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The Effects of a Computerized Information Management 
System on Classification, IEP and LRE Decisions 



Abstract 

This study evaluated the effects of a computerized management information system developed for 
special education administrators (IASP). The IASP generated information to guide staff development and 
in-service training. Effects were measured by program improvement in compliance procedures, the quality 
of IEP and LRE decisions, and the validity of classification decisions. 

To determine the district's baseline status regarding errors in the assessment, classification, and IEP 
process, two data collectors conducted assessments of special education files of students who had been 
classified as learning disabled or intellectually handicapped using three validated expert systems. Two of 
the systems, Class. LD2 and Class. IH, were classification systems designed to provide a second opinion 
regarding the appropriateness of a specific classification. The third system, Mandate Consultant, gave 
advice on compliance with special education admission processes. Based on the results of the evaluation, 
the district administrators and staff designed several interventions to remediate the errors. Staff 
development interventions varied from group (e.g., IEP in-service) to individual (e.g., reviewing the 
classification process for learning disabilities). Length of staff development interventions ranged from 30 
minutes to 3 hours. 

Follow-up data collected through the IASP process indicated that the district interventions were not 
effective. As a result, the district and project staff designed a "mentor team'' approach for processing all 
students newly referred for special education assessment. The teachers selected as mentor teachers served 
as advisors to their peers and received release time and training for their roles. Four randomly selected 
elementary schools served as an experimental group and three secondary schools were assigned as a 
comparison group. The mentor teachers were trained in five, 3-4 hour sessions conducted by project staff 
and district personnel. When a student was referred, a mentor teacher reviewed all requisite data with the 
special education teacher and processed the data with the appropriate expert system. This revised 
procedure ensured that the special education teacher participating in the child study team had been briefed 
by a "Mentor" before participation in decision-making processes. 

Following intervention (i.e., the use of mentor teachers), data collectors again evaluated files of 
students who had been newly classified as Learning Disabled (LD and Intellectually Handicapped (IH). 
Results of the study indicated that the intervention was highly effective in reducing the total number of 
errors in the assessment, classification, placement, and IEP process in the experimental schools, while the 
errors in the comparison group remained unchanged. Further, this reduction in errors occurred across all 
experimental schools, all types of errors, and both the LD and IH handicapping conditions. 

The findings strongly suggest that the combination of expert systems technology and a special 
education administrative data base created a powerful support system for progressively improving the 
quality of decisions. These decis'ons can have life-long implications for students being assessed for special 
education. The decisions also progressively and substantantively improved district staff development 
practices. 



INTRODUCTION 



One important application of computers for education administration is the use of 
computerized management information systems. In 1980, staff at Utah State University 
developed and validated MONITOR, a computerized management tool designed to evaluate 
students' progress through: (a) placement, (b) assessment, (c) DSP development, and (d) annual 
review and re-evaluation activities. The findings from the MONITOR project provided evidence 
that computerized information systems could be developed to support special education 
administration (Whitney & Hofmeister, 1983). 

According to a survey by Buirello, Tracy, and Glass man (1983), 95 percent of special 
education administrators were highly interested in the implementation of computerized 
information management systems. However, until recently the applications and scope of such 
systems have been limited. The majority of systems have focused on (1) general record keeping, 
(2) IEP generation, (3) electronic mail systems, and (4) federal and state funding requirements 
for funding and program evaluation. 

During the last five years, there has been a significant shift in the focus of computerized 
management systems. Previously, the question of importance was, "Should we implement 
computerized information systems in education?" Now researchers and educators are asking, 
"How do we most effectively design and implement such systems?" This shift in priorities has 
been facilitated by several factors, including (1) demonstrations of a variety of computerized 
information systems (Hayden, Vance, & Irvin, 1982; Hofmeister, 1982; Hofmeister, 1984; 
Ragghianti & Miller, 1982; Whitney & Hofmeister, 1983); (2) validation and effectiveness studies 
that identified strengths and weaknesses in computerized information systems (Demchak, 1986; 
Jenkins, 1987; Ryan & Rucker, 1986); and (3) increased use of computerized information 
management across educational administration, rather than just special education (Hofmeister, 
1984). 

In one such example, Ryan and Rucker (1986) found that computerized IEP program 
generated IEPs quicker and more efficiently and that teachers had a more favorable attitude 
toward the IEP for instructional planning than did teachers using non-computerized IEP systems. 
In addition, use of the computerized IEPs was more cost-effective than non-computerized IEP 
systems. Similar results of studies evaluating computerized IEPs have been reported. In a study 
by Jenkins (1987), the experimental group using computer-generated IEPs took significantly less 
time to develop their IEP's and that the quality of the computer-generated IEP was superior to 
the hand-written IEP. 



2 



The fact that several studies support the utility and validity of computerized information 
systems does not mean that all systems are practical and/or effective. In a related study, Bennett 
(1984) noted several administrative misconceptions regarding the use of computerized information 
systems. First, all automated special education information management systems will not save 
money, and some may actually generate increased paperwork and expensive redundancy. 
Second, automated systems do not automatically correct confusions and misinterpretations in 
information. In fact, some may serve to "institutionalize" rather than remove existing problems. 
Finally, an automated system will not run by itself. Computerized information management 
systems require individuals with the expertise and know-how to. operate these systems and to 
evaluate the output. A substantive investment in staff training is needed. 

Ryan and Rucker's (1986) data-based study was generally supportive of computerized IEP 
process. However, the researchers identified several questions that remained to be addressed: 

1. Did time saved in the use of computerized IEP's translate into increased instructional 
time spent with students? 

2. How can the computerized IEP process enhance the quality of IEPs? 

3. How can computerized information processing ensure the "efficacy of the IEP" as a 
functional, individualized blueprint of the intervention the child receives? 

In summary, there are clear trends in the use of computerized information management systems. 
The potential contribution of computerized systems has been well documented and recognized and 
researchers are beginning to address such critical issues as quality development and 
implementation as it relates to improved services for children with disabilities. 

Expert Systems Technology 

Sowizral and Kipps (1986) describe an expert system as a "rule-based artificial intelligence 
application program for doing work that requires expertise." They further identify the 
components of an expert system as a knowledge base with data and an internal engine that 
manages the interaction. Expert systems are designed to be interactive and typically engage the 
user in a dialogue that parallels the type of conversation that a person might have with an expert 
consultant. The computer is programmed to ask the user questions to clarify the problem or 
situation (Barr & Feigenbaum, 1981). The system then combines the information with facts and 
rule-based logic in the knowledge base to produce advice regarding a particular problem (e.g. 
appropriateness of classification). 
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For example, MYCIN is a well-known medical system for physicians that led to instructional 
applications (Davis, Buchanan, & Shorthiffe, 1975). With MYCIN the user enters information 
into the computer regarding the characteristics of the patient's symptoms (e.g., temperature, 
blood count, etc.). The computer matches the patient's data with information pertaining to 
characteristics of bacterial cultures and then, based on programmed logic presents a disease 
diagnosis. The MYCIN data base was later used in an intelligent computer-assisted instruction 
program called NEOMYCIN (Clancy & Letsinger, 1981) to teach physicians to diagnose 
bacteriological diseases. 

Special Education Application of Expert Systems 

Until recently there has been little application of expert system technology to the field of 
education (Hofmeister & Ferrara, 1986). However, with the increased power and availability of 
computer hardware and the gains in artificial intelligence, the development of expert systems for 
educators became feasible. 

Researchers at Utah State University's Artificial Intelligence Research and Development Unit 
(AIRD) have developed various expert systems designed to provide special educators with 
recommendations regarding a student's eligibility for special education. Hofmeister and Ferrara 
(1986) developed Class.LD2, an expert system that provides a second opinion on the eligibility 
of students with learning disabilities. The development of Class LD2 was followed by the 
development of Mandate Consultant, an expert system that gives advice on compliance with 
special education admission processes and on IEP procedures. Class .IH, an expert system 
similar to Class.LD2, provides a second opinion regarding the appropriateness of an intellectually 
handicapped classification. Both Mandate Consultant and Class.LD2 were subjected to a series 
of validation, replication, and training studies to assess the accuracy and general utility of the two 
expert systems. Results showed that the decision-making ability of the two systems performed 
as well as the top 10 percent of the human experts (Hofmeister & Lubke, 1986; Parry & 
Hofmeister, 1986). With minor adaptations, the systems could also function as effective inservice 
training tools (Prater, 1987). In the most recent validation study, school districts using Class 
LD2 improved the quality of LD classification decisions by more than 20 percent. The 
comparison group showed no improvement (Hofmeister & Likins, 1988). 

It is clear that expert systems can improve the quality of classification decisions (Hofmeister 
& Likins, 1988) and can serve as effective inservice tools between the low and high model 
complex decision making (Prater, 1987). However, expert systems are limited in that they 
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review problems on a case by case basis and are not designed to address district-wide staff 
training and compliance issues (Thornberg, Baer, Ferrara, and Althouse, 1990). There is a need 
to develop a computer-based support for special education administration. A support system that 
could integrate the individual expert systems with a comprehensive administrative data base to 
facilitate the development of special education programs and staff. 

Intelligent Administrative Support Program (IASP) 

The development of a powerful, integrated data management system would be unique in the 
field of education. If such a system were (1) delimited to decision areas covered by previously- 
developed expert systems, (2) providing information on compliance errors to a central database, 
then, a computerized information system could be designed with both information acquisition and 
decision support functions for special education administration. 

The major purpose of this study was to conduct a formative and a summative evaluation of 
a computerized management system developed for special education administrators. Specifically, 
the "Intelligent Administration Support Program" (IASP) was designed to assist special education 
administrators in information acquisition pertaining to the district's quality of decisions and 
procedures in the classification, placement, and IEP process. Additionally, output from the IASP 
was used to support and monitor staff development activities. 
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OBJECTIVES AND RESEARCH QUESTIONS 



The delivery and improvement of any special education service is premised on the notion that 
the child is eligible for special education services. The research collected to date suggest that 
there is little to support such an assumption with students with mild handicaps (e.g., learning 
disabled, intellectually handicapped). The purpose of this project was to develop a computerized 
information system to guide the progressive improvement of special education programs in 
compliance procedures, the quality of IEP, and the validity of classification decisions. To meet 
this objective, the Intelligent Administrative Support Program (IASP) was developed and field 
tested. IASP was designed to assist special education administrators and district staff of special 
education programs. 

Objectives 

Specific objectives of the project were: 

1. To determine if a district could use the output of the IASP to design and monitor 
effective forms of staff programming and training. 

2. To determine if use of the IASP data in conjunction with district interventions would 
result in a reduction of errors with state and federally mandated procedures in the areas 
of classification of mildly handicapped students. 

Research Questions 

Learning Disabilities 

1.1 Relative to pre-implementation, did the IASP process result in a reduction of the number 
of overall errors in classification of learning disabled students? 

1.2 Relative to pre-implementation, did the IASP process result in a reduction of the number 
of decision errors in classification of learning disabled students? 

1.3 Relative to pre-implementation, did the IASP process result in a reduction of the number 
of suspect measures used in the classification of learning disabled students? 

1.4 Relative to pre-implementation, did the IASP process result in a reduction of the number 
of procedural errors in classification of learning disabled students? 
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Intellectually Disabled 

2. 1 Relative to pre-implementation, did the IASP process result in a reduction of the number 
of overall errors in classification of intellectually disabled students? 

2.2 Relative to pre-implementation, did the IASP process result in a reduction of the number 
of decision errors in classification of intellectually disabled students? 

2.3 Relative to pre-implementation, did the IASP process result in a reduction of the number 
of suspect measures and procedural errors in classification of intellectually disabled students? 

IEP Procedures 

3. 1 Relative to pre-implementation, did the IASP process result in a reduction of the number 
of overall compliance errors associated with federal and state mandated IEP procedures? 

3.2 Relative to pre-implementation, did the IASP process result in a reduction of the number 
of federal and state mandated procedure errors prior to the IEP meeting? 

3.3 Relative to pre-implementation, did the IASP process result in a reduction of the number 
of procedural errors associated with the development of the IEP document? 

3.4 Relative to Pre-implementation, did the IASP process result in a reduction of the number 
of errors associated with the federal and state mandated procedures in an IEP? 
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METHODS 

The methods section includes a discussion of the (a) subjects, (b) procedures, (c) instruments 
and data collection, (d) research design, (e) procedures, and (f) analysis of data. The following 
is a detailed description of each portion for accomplishing the project objectives. 

Subjects 

The target population for this study was all public school districts in Utah. The accessible 
population was limited to Ogden City School District in Ogden, Utah, an inner-city district with 
the largest minority population of any school district in the state. The district also had a large 
transient population. 

The seven schools were randomly selected from all public schools in the Ogden School 
district. Four elementary schools were assigned to the experimental group and three secondary 
schools served as a comparison group. Elementary schools were selected to serve in the 
experimental group because of the extent of interaction and cooperation that project staff required 
to access files and to facilitate the performance feedback process within the district. In addition, 
other elementary schools were not selected to serve as a control sample because of potential 
contamination resulting from the "mentor teacher" model. The "mentor teacher" model in which 
an experienced, well-qualified teacher shares his/her expertise with more inexperienced teachers, 
was emphasized and practiced throughout the district and state-especially at the elementary level. 

The Intelligent Administration Support Program GASP) was designed to assist special 
education administrators in information acquisition pertaining to the district's quality of decisions 
and procedures in the classification, placement, and IEP process. Additionally, the IASP 
consisted of an expert system as well as a database program and special education student "cache" 
files generated by other validated expert systems, Class .LD2, Class. IH, and Class.MC. An 
overview of the IASP system is summarized in Table 1. 

The development and validation of the IASP occurred in four major project design phases: 
(a) content clarification and system design, (b) development and field test of the prototype and 
revisions, (c) main field test with program evaluation, and (e) final revisions based on field test 
data (see Figure 1). Each of the phases will be discussed in terms of specific activities. 
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Table 1. An Overview. 

The Intelligent Administrative Support Program (IASP): An Overview 



Separate 
Expert 
Systems 
Provide 



Information on 
the quality of 
decisions and 
procedures 
which can be 
cross- 
referenced with 



An intelligent 

database which 

provides the 

special 

education 

administrators 

with 



Individual 
staff, schools, 
K-12 divisions, 
and program 
units to 
provide 
direction to 



Program 
evaluation and 
staff 

development 
activities 



Expert Systems 
Include: 

1. Mandate 
Consultant 

2. Class.LD 

3. ClassJH 



Information 
Would be 
Provided on: 



1. 



3. 



Compliance 
with proce- 
dures to 
admit 
students to 
special 
education, 

The quality 
of IEPs and 
LRE 

decisions, 
The 

validity of 
classifica- 
tion 

decisions 



The Intelligent 
Database 
Consists of: 



1. 



2. 



A district 

wide 

relational 

database, 

and 

An expert 
system to 
evaluate the 
sampling 
and retrieve 
the infor- 
mation in 
keeping 
with user 
needs 



Examples of 
Units of 
Analysis 
Include: 

1. Individual 
staff 

2. Individual 
schools 

3. K-12 units 
e.g., 

elementary, 
middle 
school, 
high school 

4. Special 
education 
program 
units, e.g., 
mildly 
disabled, 
severely 
disabled. 



Program 
Improvement 
Would be 
Aided by: 

1, IASP data 
-on pre- 
scriptions 
for staff 
develop- 
ment, 
(e.g., 
which 
skills need 
to be 
mastered 
by which 
staff), and 

2. Data 
analyzing 
the impact 
of staff 
develop- 
ment 

experiences 
and related 
program 
changes. 
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Figure 1. Maior Project Phases. 



Phase 1 

Content 
Clarification and 
System Design 




Phase 2 

Development and 
Field Test of 
Prototype and 
Revisions 



Phase 3 




Phase 4 

Revisions Based 8 

on Field Test 0 

Data 8 



Content Clarification and System Design 

Before designing a new expert system, it was necessary to review the literature to identify 
recent developments and practices. In addition to the review for the proposed project, a group 
of professionals from special education met to substantiate and clarify the problems to be 
addressed by the project. The group included: (a) a staff member from the Utah State Office 
of Education, (b) a special education administrator from a local education agency, (c) a Utah 
State University (USU) faculty member from the Special Education Department, and three staff 
members from the USU Artificial Intelligence Research and Development Unit. 

The professionals required several meetings to clarify and define answers to the following 
system questions: 

a. What type of problem should the system address? 

b. What type of information output should the system provide? 

c. Who should use the system? 

d. What computer software and hardware hold the most promise? 

e. How should the system be used? 

Prototype Development and Revisions 

Several prototypes were developed and revised during the formative evaluation. While initial 
stages of development focused on individual components or modules, the latter stages addressed 
the relationships and interactions among the various modules and overall system performance. 
Some of the key modules included in the system were: (a) the central structure of the data base, 
(b) the coordination and revision of "feeder" expert systems, (c) the information acquisition and 
user interface, (d) the expert system for the intelligent front end to the data base, (e) the report 
generator and information retrieval, and (f) the system documentation. 

To increase the content validity of the IASP outcomes, the project staff and district 
coordinators reviewed the content of the IASP outputs. Any outputs that were unclear were 
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modified in that meeting to meet the administration's needs. The format for the summary report 
was modified substantially to facilitate understanding. Some of the modifications included: (a) 
the addition of headings and columns to better organize the data, (b) the addition of frequencies 
specific to student and school, (c) a full description versus an abbreviated version of the specific 
problem or violation of state and federal rules, (d) a list of the students' files that were evaluated, 
and (e) the certainty factor (cf) that was associated with the likelihood that a student should be 
classified. Examples of an IASP summary report are included in Appendix A. 

The primary design specifications that served to guide and evaluate the product development 
during the formative evaluation period were (a) the software development systems and languages, 
(b) the hardware requirements for field use and central information management, (c) the user 
information regarding collection and interpretation needs, (d) dissemination and maintenance costs 
of total system, (e) the nature of the availability and validity of information collected, (f) the 
practicality of the content and format of reports generated by the system, (g) the potential for 
political threats, generated by such issues as staff evaluation, (h) the potential for legal threats, 
generated by possible conflicts with state and federal regulations, (i) the potential for enhancing 
special education program credibility with staff, other administrative units, the community, and 
special education students and their families, (j) staff development costs associated with the use 
of the system, and (k) the relationship between the IASP and district administrative and program 
evaluation policies. 

Main Field Test and Program Evaluation 
Dependent Variables 

The dependent variables for the main field test included, (a) overall errors, decision errors, 
suspect measures, and procedural errors in classification of learning disabled students, (b) overall 
errors, decision errors, and procedural errors or suspect measures in classification of intellectually 
disabled students, (c) overall compliance errors, errors prior to thelEP meeting, errors associated 
with the development of the IEP document, errors associated with the IEP meeting. More 
detailed information and specific examples of each dependent variable is listed in Table 2. The 
dependent variables were identified by the research literature reviews and combined with the 
recommendations of the special education administrators and staff of Ogden City Schools. 
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Data Collection 

As a part of the systems design, it was necessary to assess the district baseline status 
regarding errors in the assessment, classification, and IEP process. To identify procedural errors, 
two data collectors conducted assessments of students special education files who had been 
classified as learning disabled, and intellectually handicapped within the current school year. 
Three to four student files were randomly selected from each handicapping category per school. 
To facilitate the random selection process, a roster of newly classified special education students 
was generated by the district office for each school from which students* names were randomly 
drawn. Pre-intervention data collection occurred during Spring of 1990 prior to the use of the 
IASP and district interventions. 

Following intervention, the same data collectors again evaluated files of students who had 
been newly classified as learning disabled and intellectually handicapped during the Fall of 1990 
to Spring of 1991. The student's files were randomly selected from the same target population. 
As in the prior evaluation, 3-4 student files from each handicapping condition for each school 
were evaluated. The evaluation procedures and expert systems used were the same as those 
described for pre-intervention. 

A total of 383 special education students* files were reviewed during the pre- and post- 
assessment period. Of those files, 179 were files of students who had been classified as Learning 
Disabled (LD), and 71 files were those of students who had been classified as intellectually 
disabled (IH). The expert system, Mandate Consultant (MC) was used to review 133 files for 
IEP and due process procedural errors. These files were randomly selected from the LD and IH 
files previously reviewed. The total number of files reviewed in the comparison and experimental 
groups appear in Table 3. 
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Table 2. Description of errors found in files of students who had been classified as learning 
disabled (LB) or intellectually handicapped flffi, 



| Type of Errors J 


Examples jj 


1 Overall Errors 
Decision Errors 


• Total number of errors per student file 

• Misclassification or Non-LD explanations 

• Poor-classification (i.e., less than SO percent confidence that the 
student can be classified LD or IH) 


| Suspect measures 


• Old, musing, or incomplete result(s) of vision or hearing test 

• Inappropriate or unqualified tests (e.g., IQ test or academic test 
must be approved by the state guidelines for LD classification) 

• IQ score is less than 75 for LD, or greater than 75 or less than 60 
forffl 

• Missing documents required by state (i.e., Classification 
Summary, Scrams, IQ tests, parental permission, referral for 
evaluation, etc.) 

• Missing adaptive test for IH 

• Academic achievement is not assessed 


Procedural Errors 


• Classroom observation^) is missing or incomplete 

• Prereferral interventions are missing or incomplete 

• More than 30 days lag between parental permission and initial date 
of assessment 

• Signatures and dates on protocols in pencil 

• Documentation of prior notice is missing 


Overall Errors 


* Total number of errors per student 


Errors prior to IEP 
meeting 


• IEP documents) missing 

• IEP documents) not completed within 30 days of classification 
date 

• More than 30 days between parent permission and student 
evaluation 

• Missing or incomplete prereferral intervention^) 


Errors associated 
with IEP meeting 


• Absence of regular teacher 

• Absence of public agency 

• Absence of parents 

• Absence of special education teacher 

• Lack of acceptable assistance to the parent whose primary 
language is not English | 


Errors associated 
with the 
development of 
IEP documents 


• The IEP document is not reviewed within one-year 

• Missing signatures or titles of team members 

• Signed after services are initiated 

• Failed to state or incompletely stated demographic information, 
student's involvement in regular education or special education, or 
related services 

• Missing or incomplete annual goals, short term objectives, 
evaluation procedures and/or schedules, or present levels of 
performance statements) 
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Table 3 

Number of Student Files bv Group and by Handicapping Condition 



| Group 


Total 


LD 


IH 


|| 

MC I 


Experimental 


225 


111 


40 




Comparison 


158 


68 


31 


59 


Total 


383 


179 


71 


133 



Table 4 presents the number of student files reviewed in the experimental group by school. 
The data indicate that there was an unequal distribution across schools of the files classified as 
intellectually disabled. Only three students were classified as intellectually disabled in School 4, 
while School 2 classified 21 students classified as intellectually disabled. Furthermore, the three 
students' files in School 4 were classified during pre-intervention period. There were no students 
newly classified as intellectually handicapped during the post-intervention period. The lack of 
files and their unequal distribution across schools limited generalization of the results. 
Table 4 



Number of Student Files hv School and bv Disabled Condition in Experimental Group. 



School 


LD 


IH 


MC 


1 


39 


7 


22 


2 


23 


21 


17 


3 


31 


9 


16 


4 


18 


3 


19 


Total 


111 


40 


74 



Table 5 shows the distribution in the experimental group by school year. Files evaluated 
during pre-intervention were selected from files of students classified during the last three school 
years (i.e., 1987-88, 1988-89, and 1989-90 school year). The post-intervention files consisted 
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Table 5 



Number of Student Files h v School Year and bv Disabled Condition in Experimental Group. 



| School Year 


LD 


IH 


MC 1 


1987 - 1988 


3 


8 




1988 - 1989 


43 


13 


10 


1989 - 1990 


37 


6 


25 1 


1990 - 1991 


28 


13 


32 I 


| Total 


111 


40 


74 I 



only of newly classified students (i.e. 1990-91) school year. A total of 152 files were reviewed 
during the pre-assessment period, of those files, 83 files were LD, 27 files were IH, and 42 files 
were MC. During the post-assessment period, a total of 73 files were reviewed. Of those files, 
28 files were LD, 13 files were IH, and 32 files were MC. 

To conduct the file assessment, the data collectors used three expert systems: 

Class.LD2 is an expert system designed to provide a second opinion regarding the 
appropriateness of a learning disabilities classification based on Utah and Federal regulations 
related to P.L. 94-142. 

Class.IH is an expert system designed to provide a second opinion regarding the 
appropriateness of an intellectually disabled classification based on Utah and Federal regulations 
related to P.L. 94-142. 

Mandate Consultant (MC) is an expert system which focuses on administrative issues in IEP 
development. It is designed to provide advice regarding the individualized education program 
development procedures mandated by P.L. 94-142 and Utah rules and regulations. 

Both Mandate Consultant and Class.LD2 have been subjected to a series of validation, 
replication, and training studies to assess the validity, generalizability, and general utility of the 
two systems. When the decision-making ability of the two systems was compared with that of 
human experts, the systems performed as well as the top 10 percent of the comparison population 
(Parry & Hofmeister, 1986; Hofmeister & Lubke, 1986; Martindale, Ferrara, & Campbell, 
1988). It was also found that with some adaptations, the expert systems could be used as 
effective inservice training tools because of their ability to model complex decision-making 
(Prater, 1987). In the most recent validation study involving Class.LD2, using randomly 
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assigned treatment and comparison groups, the quality of LD classification decisions improved 
by more than 20 percent in school districts using Class.LD2. The comparison group showed no 
improvement in the quality of LD classification decisions (Hofmeister & Likins, 1988). 

Conducting a Consultation 

To use one of the expert systems, the data collectors examined the student's folder 
independently and answered questions that were generated by the expert system pertaining to the 
specific handicapping condition and the classification process. The consultation typically required 
15-20 minutes. At the end of the consultation, the expert system provided advice and a rationale 
based on the data entered. As a part of the consultation process, each expert system generated 
a memory cache, or individual file on the characteristics of the particular case. The cache files 
contained a record of ail the findings, inferences, and conclusions for each student consultation. 
In the case of Class.LD2, the cache could have contained up to 170 pieces of information on 
student characteristics, instructional history, the classification decision, and other advice. The 
cache files were read and evaluated by the IASP to generate a large database. The database 
contained final conclusions, intermediate conclusions, the data upon which the conclusions were 
based, and a specific list of the type and frequency of classification and placement errors in all 
student files. 

Reliability 

To assess reliability, two data collectors conducted independent assessments of at least three 
Of the same students' files for each disability condition and school using the expert systems. The 
consultation outcomes were then compared on a question by question basis to determine 
agreement. Percent agreements were calculated by dividing the number of agreements by total 
number of agreements and disagreements. The data collectors maintained an average of 90 
percent agreement across all student files reviewed, with a range of 80 percent to 100 percent. 

To verify the data generated by the IASP, the project staff randomly selected 20 percent of 
student files, and compared the IASP data with written summary reports from selected files. 
Conclusions generated by the IASP did not differ significantly from the written summary reports. 
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Interventions 

Project staff presented Ogden administrators and staff with the data from the pre-assessment. 
The frequency and types of procedural errors for each school were summarized and discussed in 
relation to staff training needs. It was determined that each district coordinator would use the 
data to plan and implement specific staff development interventions with individuals they 
supervised. The district coordinators were asked to maintain a record of the type of interventions 
that was implemented and to provide descriptive information regarding the outcome and follow-up 
measures (see Appendix B). In addition, on-going data summaries from the IASP assisted the 
district personnel in monitoring the interventions. A summary of initial interventions that were 
implemented by the Ogden district staff is presented in Table 6. Interventions ranged from group 
(e.g., IEP inservice for entire special education staff) to individual (reviewing the LD 
classification process). One district-wide memo was sent to clarify a procedure regarding hearing 
test results. Length of intervention ranged from 30 minutes to 3 hours. Follow-up data from the 
IASP indicated that the district interventions were not effective. As a result, a higher intensity 
staff development intervention was designed by the district and project staff. 

The district's special education director and coordinators selected six special education 
teachers to serve as a "mentor team". The six teachers had an average of ten years teaching 
experience. Five of the teachers were resource teachers and one was the teacher of a self- 
contained unit serving children with behavior disorders. All of the teachers were considered 
mentor teachers and frequently supervised or trained peers in the district. 

The purpose of the mentor team was to screen and review the assessment and classification 
process for all students newly referred for special education in the elementary program. Each 
mentor was equipped with a portable computer and the following expert systems: Class.LD2, 
Class.IH, and Mandate Consultant. Prior to the beginning of school, the mentor team received 
extensive training. During five, 3-4 hour training sessions conducted by project staff and district 
coordinators. Content of the training included: (1) review and discussion of the state and federal 
rules for qualifying students under each handicapping condition; (2) demonstration and hands on 
practice with the expert systems using student files; (3) overview of computer basics and an 
introduction to DOS (disk operating system). In addition to training, the mentor team redesigned 
all district classification forms (e.g. LD, IH, and BD) to facilitate compliance with state and 
federal rules and developed a classification tracking form (see Appendix C) to assist the team and 
the district in monitoring the student's progress from referral through the assessment, 
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Table 6. Oeden District Initial Interventions. 



1 *VllTO At A f*A9 


Intervention 


nuw 

Long 


itesuiis 


rouow-up 




Group/Ind 


Description 






LD 

Classification 


Individual 
one-on-one 


Check on 
classification 
process/conformity 
with district policy 


30 min 


6 files reviewed 
2 of 6 correctly 


Additional 
Service 


Assessment 


Individual 
one-on-one 


Annronriate u^e of 
measures of adaptive 
behavior 


30 min 


To be 
determined 


Special Ed. 
will 

'lllUllllXll 

future IH 
referrals 


LD 

Classification 


Individual 
one-on-one 


Check on 
classification 
process/follow-up to 
previous visit 


30 min 


2 files reviewed 
1 of 2 correctly 
completed 


continue to 
monitor 


IEPs 


Group 
(N=20) 


District Inservice 
including: 

a. current levels of 
performance 

b. annual goals and 
objectives 

c. monitoring 


3 hn 


Teachers turned 
in an IEP for 
critique must 
meet 90% 
criterion 


A second 
IEP by 
those who 
did not 
meet 
criterion 
was 

completed 
and 

evaluated. 


The Special 
Education 
Referral 
Process 


Group 
(N=20-30) 

per 9vnooi 


Faculty Inservice at 5 
schools issues 
discussed: 

a. appropriate 
referrals 

b. pre-referral 
interventions 

c. accuracy of vision, 
hearing, and 
bilingual screening 
data 


lhr 
IS min 


To be 

determined by 
appropriateness 
and complete- 
ness of future 
referrals 




Hearing Test 
Results 


Individual 


Memo, sent to all 
special personnel 
informing them of 
procedures to access 
results 


N/A 


Check future 
files to 
determine if 
results are 
recorded 




Continue to 
monitor 
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classification, and placement process. If a student did not qualify, the multidiscipUnary team 
recommendations at the school were recorded and follow-up data were noted. Each mentor was 
assigned to serve three elementary schools in the district and was released from their teaching 
duties two half days a week. A permanent substitute teacher was hired to fill in for the mentor 
during this time. 

As a team member, the mentor would visit each assigned school once a week. When a 
student was referred for special education, the special education teachers were directed to contact 
their mentor teacher who would review the screening data (e.g. vision and hearing tests, 
bilingust, etc.) with the teacher and then direct the teacher to collect specific data. After the 
assessment was. completed, the resource teacher would again contact the mentor teacher who 
would run a consultation using the appropriate expert system and specific student data. The 
results of the consultation and program advice were presented in an expert system summary 
report and discussed with the special education teacher (see Appendix D). The teacher was then 
directed to share the results with the multidiscipUnary team as a part of the assessment process. 
Final classification decisions were always determined by a school's multidiscipUnary team, rather 
than the summary report. The expert system summary report was considered a second opinion 
to facilitate the decision-making process. To facilitate implementation, district staff designed 
flowcharts of the mentor team process for each classification. The LD and IH flowcharts are 
included in Figures 2 and 3 respectively. 

Several follow-up meetings were held throughout the school year with the project staff and 
mentor team to review problems and to provide support. Problematic issues included: (1) faculty 
complaints regarding length of time to classify a student, (2) program bugs, (3) multidiscipUnary 
teams overriding state guidelines with little or no documentation, and (4) insufficient or 
incomplete data to process an expert system consultation. 

Following intervention, the same data collectors again evaluated files of students who had 
been newly classified as learning disabled and intellectually handicapped during Fall of 1990 to 
Spring of 1991 . The data collection and evaluation procedures and expert systems used were the 
same as those described for pre-intervention. 
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Classification Process for Students with a 
Suspected Handicap of Specific Learning Disabilities 



Written prior notice 




Referral 




Permission to evaluate 





Contact district mentor team member to notify of case. The 
following information will be necessary: 

1. Date 

2. Student name 

3. Teacher/school 

4. Date of written prior notice and permission to evaluate 

5. Suspected handicap 

6. Vision and hearing screening dates and results 



Mentor feicher completes information 
checklist and instructs teacher to proceed with 
usual evaluation/information gathering 
procedures. 



Follow usual procedures for evaluating and gathering 
pertinent information regarding the student. Contact the 
school psychologist on new referrals and make arrangements 
for a WISC-R or WPPSI to be administered. This will save 
teacher testing time and duplication of effort. 



Upon completion of the evaluation/information gathering, make a second 
contact with the mentor teacher to inform him/her that you are ready to 
have the case run through the LD expert system. 



Mentor teacher comes to the school during his/her regularly scheduled time and 
inputs the information gathered mm the LD mpm system computer program 
which will provide m opinion km&mty factor) as to whether or not the student 
should be considered teaming disabled. 



Conduct a classification team meeting to discuss testing results 
with parents. Note: If the student is eligible for service the IEP 
may also be written at this time. 



Figure 2 . Classification process for LD 
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Classification Process for Students with a 
Suspected Handicap of Intellectually Handicapped 



Written prior notice 

Referral 
Permission to evaluate 



Contact mentor teacher to notify of case. The following 
information will be necessary: 

1. Date 

2. Student name 

3. Teacher/school 

4. Date of written prior notice and permission to evaluate 

5. Suspected handicap 

6. Vision and hearing screening dates and results 



Mentor teacher completes information checklist and instructs 
teacher to proceed with usual evaluation/information gathering 
procedures (Reminding teachers that an adaptive measure is 
required). 



Follow usual procedures for evaluating and gathering pertinent infor- 
mation regarding the student. Note: An adaptive measure (i.e. 
VINELAND, AAMD) is required. When all information is gathered, 
contact the district psychologist to request a WISC-R/WPPSI and to 
get the adaptive scored. 



When you receive a statement of findings back from the psychologist, 
make a second contact with your mentor teacher to schedule a time to run 
the information through the IH expert system. 



Mentor teacher conies to the school during his/her regularly scheduled time and 
inputs the information gathered into the HI expert system computer program 
which will provide an opinion (certainty factor) as to whether or not the student 
should be considered intellectually handicapped. 



Conduct a classification team meeting to discuss testing results 
with parents. Note: If the student is eligible for service the IEP 
may also be written at this time. 



Figure 3 . Classification process for IH 
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Research Design 

An interrupted multiple time-series analysis design (Glass, Wilson, & Gottman, 1975), 
employing a non-equivalent comparison group, was used to investigate the effect of the IASP on 
the dependent variables. A time-series design was selected to analyze the data in the study for 
a number of reasons. First, student files evaluated prior to and after intervention were not the 
same. Because individual files have different characteristics (i.e., the degree of disability, 
schools, grade, or teachers, etc.), a normal per- and post-test design was not considered 
appropriate. 

Second, the most basic time-series experimental design involves some number of repeated 
observations, "0", of an outcome variable across time with an intervention, "I" introduced 
between two observations: A change in some property of the observations (i.e., level and trend) 
that coincided with the "I" might have been the effect of "I" on the outcome variable intervention 
(Campbell, 1969; Glass, Et al., 1975). As a result, time-series designs and associated data 
analysis techniques were recommended to facilitate more accurate interpretations of the 
intervention effects than a one-group pretest and posttest design (e.g., changes in level, trend, 
or both). 

To conduct the time-series design, successive observations were collected prior to and after 
interventions from the experimental and comparison groups to assess the impact of the IASP 
process. The comparison group, consisting of three secondary schools, was used to examine 
historical threats to the design. Multiple factors, such as changes on state guidelines, teachers' 
or paraprofessionals' attitude, number of special education students, or support from other 
teachers, could effect the outcome measure. To control for these potential factors or rival 
hypotheses, the multiple time-series design compared the data from the experimental group, 
consisting of four elementary schools, with that of a comparison group (three secondary schools). 

The multiple group time-series design is diagrammed in Figure 4. The notation employed 
is as follows: "I" depicts an intervention into the sequence of observations. The compliance 
errors per student classified as learning disabilities are represented by OA, the compliance errors 
per student classified as intellectually disabled are represented by OB, and the compliance errors 
with IEP development per student served in special education are presented by OC. The numbers 
followed after notations represents the number of files reviewed. Experimental and comparison 
observations are separated by a dashed line. 
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0A-10A-2 .. OA-82 OA-83 I OA-84 . OA-Ill 
OB-1 OB-2 . . OB-26 OB-27 I OB-24 . OB-49 
OC-1 OC-2 . . OC-41 OC-42 I OC-43 . OC-74 



OA 1 OA-2 . . OA-49 OA-50 I OA-51 . OA-68 
OB-1 OB-2 . . Ob-18 OB-19 I OB-20 . . OB-31 
OC-1 OC-2 . . OC-40 OC-41 1 OC-42 . OC-59 



Figure 4. The multiple group time series design. 

Data Analysis 

The means and associated standard deviations of the dependent variables during pre- and post- 
interventions were computed. The means and standard deviations were used to identify degrees 
of variation in the observational data associated with the IASP. In addition, effect sizes were 
computed with the pre- and post-assessment means of each dependent variable in conjunction with 
the p-values to determine the magnitude of the difference, between error means of the dependent 
variables during pre- and post-interventions. 

Chow-tests with dummy variables were then used to analyze the data. The Chow-test is a 
statistical technique that tests differences between two or more regression lines (Gujarati, 1970). 
However, according to Gujarati (1970), the Chow-test doesn't explicitly tell whether the 
difference between two or more regressions is due to the intercept or the slope, or both. To 
specify the sources) of the difference, Gujarati (1970) introduced the use of the dummy variable 
(D) in the regression equation. For example, D is assigned 0 for the observations conducted 
prior to interventions and 1 for observations completed after interventions. The dummy variable 
approach identifies the difference between two regression lines and whether it is due to the 
intercept, or the slope, or both. A significant result (i.e., t-associated statistics of intercept or 
slope, or both) would indicate that the profile of the variables differed significantly from pre- to 
post-assessment due to the reduction of errors (intercept) or the increased/decreased trend (slope), 
or both. 
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Results and Discussion 

Data were gathered from special education student files using three validated expert systems 
and analyzed on a pre/post intervention basis to assess the impact of the Intelligent Administration 
Support Program (IASP) process on the compliance procedures, the quality of IEP and LRE 
decisions and the validity of classification decisions in seven schools in Ogden School District. 
The raw data collected during pre- and post-intervention are included in Appendix F. This report 
presents the results of the evaluation as it relates to the research questions presented in the 
Introduction. The first section discusses the findings for the files of students classified as learning 
disabled, the second section describes the findings for the files of students classified as 
intellectually disabled, and the third section discusses the findings relative to federal and state 
rules mandated for IEP procedures. 

Learning Disabilities 

Evaluation Question 1.1: Relative to pre-implementation, did the IASP process result in a 
reduction of the number of overall errors in classification of learning disabled students? 

Descriptive Statistics 

Mean overall errors, mean decision errors, and mean of suspect measures and procedural 
errors per file of students classified as learning disabled during post-intervention period are 
presented for experimental and comparison groups in Figure 5, 6, and 7 respectively. Following 
intervention, the difference of mean overall errors between groups were 4.49 errors per student 
file, with an average of 0.18 errors per student file in the experimental group and an average of 
4.67 errors per student file in the comparison group. Figure 6 shows a marked difference of 
mean decision errors (0.33 errors per student file) between experimental (0. 1 1 errors per student 
file) and comparison (0.44 errors per student file) groups. The mean errors of suspect measures 
and procedural errors are presented in Figure 7. The mean errors per student file for the 
experimental group was 0.08 as compared to 4.22 errors per student file for the comparison 
group. Further analyses of the different types of errors per student files are displayed by 
individual schools in experimental groups in Figures 8, 9, 10, and 11. Data show that the files 
of students that were classified as LD during the post intervention had a smaller number of errors 
than those previously assessed. 
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LD 



IH 



MC 



Total 




6 8 10 

Mean Errors per Student 



12 



14 



16 



Exp. Group 
(N = 41) 



] Comp. Group 
(N = 30) 



Figure g. Post-Intervention Data: Mean Overall Errors between Experimental and Comparison 
Group. 



LD 



IH 



Total 




0.6 0.8 1 
Mean Errors per Student 



1.2 



1.4 



1.6 



Exp. Group 
(N = 41) 



1 Comp. Group 
(N = 30) 



'F_iaiie_$. Post-Intervention Data: Mean Decision Errors between Experimental and Comparison 
Group. 
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LD 



IH 



Total 




4 6 
Mean Errors per Student 



10 



Exp. Group 
(N = 41) 



1 Comp. Group 
(N = 30) 



Figure 7 . Post-Intervention Data: Mean Errors of Suspect Measures and Procedural Errors 
between Experimental and Comparison Group. 
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Mean Errors per Student (N=10) 

■ 



Prior to Intervention 



3 After Intervention 



LD1: Decision errors 
LD2: Suspect measures 
LD3: Procedural errors 



Fjguie_S. Mean Errors per LD Student from School 1. 



30 



0 



1 2 3 

Mean Errors per Student (N=8) 

Prior to Intervention ||||| 

LD1: Decision errors 
LD2: Suspect measures 
LD3: Procedural errors 



] After Intervention 



Figure 9 . Mean Errors per LD Student from School 2. 



LD1 



LD2 



LD3 



Total 




0 



4.75 



Prior to Intervention 



2 3 
Mean Errors per Student (N=7) 

cz 



] After Intervention 



LD1: Decision errors 
LD2: Suspect measures 
LD3: Procedural errors 

Figure 10. Mean Errors per LD Student from School 3. 
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LD1 



LD2 



LD3 



Total 




Prior to Intervention 



2 3 4 5 

Mean.Errors per Student (N=3) 

1 After Intervention 



LD1: Decision errors 
LD2: Suspect measures 
LD3: Procedural errors 



Figure 11. Mean Errors per LD Student from School 4. 



The numbers of mean errors and associated standard deviations for the experimental and 
comparison group during pre- and post-interventions are presented in Table 7 and 8 respectively. 
Prior to the intervention, the experimental group averaged 4.63 overall errors per student file. 
The overall errors dropped to an average of 0.18 per student file during the post-intervention, 
a statistically significant decrease of 4.44 errors per student file. The number of overall 
Table 7. 

Pre- and Po st- Intervention Data Analysis for Experimental Group 



Dependent Variables 


Pre-intervention 


Post-intervention 


Effect 
Size 


p-value 


Mean 


SD 


Mean 


SD 


Learning Disabilities 


Overall Errors 


4.63 


2.60 


0.18 


0.39 


-1.71 


0.000* 


Decision Errors 


0.69 


0.76 


0.11 


0.32 


-0.76 


0.001* 


Suspect Measures 


2.47 


1.40 


0.04 


0.19 


-1.74 


0.000* 


Q Procedural Errors 


1.46 


0.90 


0.04 


0.19 


-1.58 


0.000* | 
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Table 8. 

Pre- and Post- Intervention Data Analysis for Comparison Group 



Dependent Variables 


Pre-intervention 


Post-intervention 


Effect 


p-value I 




Mean 


SD 


Mean 


SD 


Size 




Learning Disabilities 








Overall Errors 


4.94 


1.59 


4.67 


1.46 


-0.17 


0.555 


Decision Errors 


0.58 


0.67 


0.44 


0.51 


-0.20 


0.473 


Suspect Measures 


2.80 


0.83 


2.61 


0.61 


-0.23 


0.418 


Procedural Errors 


1.58 


0.73 


1.61 


0.78 


0.04 


0.872 1 



errors for the comparison group remained approximately the same with the average number 
dropping from 4.94 (pre) to 4.67 (post), a decrease of 0.27 errors per file. The mean decision 
errors for the experimental group also dropped from 0.69 to 0. 1 1 after intervention, a statistically 
significant decrease of 0.58 errors. In the comparison group, the number of decision errors 
remained unchanged, with 0.58 errors (pre) and 0.44 errors (post). 

As indicated in Table 7, the number of suspect measures for the experimental group 
significantly decreased from an average of 2.47 (pre) to 0.04 (post), an average decrease of 2.43 
errors per file. The suspect measures for the comparison group remained relatively constant (see 
Table 8) from pre- to post-intervention. 

Finally, as indicated in Table 7, there was a significant reduction in the number of procedural 
errors per student file for the experimental group after intervention. The number of errors 
dropped from 1.46 to 0.04 per student file, a significant decrease of 1.42 procedural errors per 
student file. The number of procedural errors in the comparison group remained unchanged from 
pre- to post-intervention with the average number increasing from 1.58 to 1.61 errors. 
Evaluation Question 1.2: Relative to pre-implementation, did the IASP process result in a 
reduction of the number of decision errors in classification of learning disabled students? 

Assumptions of the Chow Test: Analyses were performed to examine the assumptions (i.e., 
linearity, homogeneity of variance, normality, and independence of errors) of the regression 
procedure underlying the Chow statistic used in this study. The assumptions of linearity and 
homogeneity were examined by plotting the residuals against the predicted values and also against 
the independent variables. Corresponding graphs can be found in Appendix E. In both plots 
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there were relatively unsystematic patterns to the spread of the residuals, indicating that the 
assumptions of linearity and homogeneity of variance were met. The assumption of normality 
was checked by plotting the distribution of the residuals. The distribution of the residuals 
appeared to be fairly normal, indicating that the assumption of normality was met. Another 
assumption of independence of errors was checked by plotting the residuals against the sequence 
variable. There was not systematic pattern in those plots, indicating that the assumption of 
independence of errors was met. 

Statistical significance. The Chow Test analyses of the effect of the IASP on overall errors 
is presented in Table 9. 
Table 9. 

Analysis of the effect of the IASP on the classification of students with handicaps in Ogden 
School District. 



Dependent Variables 


Experimental Group 


Comparison Group. 


Learning Disabilities 


Intercept 


Slope 


Intercept 


Slope 




t 


P 


t 


P 


t 


P 


t 


P 


Overall Errors 


-4.916 


0.000* 


0.071 


0.943 


1.381 


0.172 


1.151 


0.254 


Decision Errors 


-2.597 


0.011* 


-0.132 


0.895 


0.784 


0.436 


0.040 


0.968 


Suspect Measures 


-4.686 


0.000* 


0.316 


0.753 


1.535 


0.130 


0.410 


0.968 


Procedural Errors 


-4.591 


0.000* 


-0.154 


0.878 


0.421 


0.675 


-1.862 


0.067 



The table presents the t-values for the intercept (i.e., reduction of the number of errors 
between pre- and post-intervention) and slopes (i.e., changes in the trend of errors across time 
between pre- and post-intervention) associated with the number of overall errors in classification 
of students with learning disablities. As indicated in Table 9, the effect of the IASP was 
significant with intercept (t = -4.916; p = 0.000), but insignificant with slope (t = 0.071; p = 
0.943) at the probability level of .05 for the experimental group. These results show that there 
was a statistically significant reduction of the number of overall errors between the regression 
lines of pre- and post-intervention for the experimental group. However, the trend of the 
regression line prior to intervention was similar to the trend of the regression line after 
intervention. In the comparison group, both intercept (t = 1.381; p = 0.172) and slope (t = 
1.151; p = 0.254) of the number of overall errors were statistically insignificant at the .05 level. 
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Educational significance. To evaluate educational significance, effect sizes were calculated. 
Table 7 displays the effect sizes (ES) between pre- and post- mean overall errors in the 
experimental group. Results indicated that there was a statistically significant reduction of the 
mean of overall errors (ES = -1.71; p = 0.000) at the .05 level between pre- and post- 
intervention. Thus, the mean number of overall errors after intervention dropped approximately 
two standard deviations below the errors prior to intervention. A similar reduction did not occur 
in the comparison group (ES = -0. 17; p = 0.555) (see Table 8). 

Statistical Significance. An analysis of the effect of the IASP on decision errors based on 
the Chow test is presented in Table 9. Data from Table 9 indicate that the reduction of decision 
errors between pre- and post-intervention for the experimental group was significant with 
intercept (t = -2.597; p = 0.011) and the change of the trend of decision errors across time 
between pre- and post-intervention was insignificant with slope (t = -0.132; p = 0.895) at the 
.05 level. Therefore, although there was a statistically significant reduction in the number of 
decision errors between the regression lines of pre- and post- intervention for the experimental 
group, there was no statistical significance between the pre and post trend lines. In the 
comparison group, both intercept and slope were statistically insignificant at the .05 level. That 
is, the number of decision errors per student file failed to decrease or increase in the comparison 
group and the trend of decision errors remained constant across time. 

Educational Significance. The analysis of the effect of the IASP between pre- and post- 
mean decision errors is presented with effect sizes in Table 7. Effect sizes in the experimental 
group showed a statistically significant reduction of mean decision errors between pre- and post- 
intervention (ES = -0.76; p = 0.001). This result indicates that the mean number of decision 
errors dropped to approximately one standard deviation following intervention. For educational 
research the difference is significant at the .001 level. Data in Table 8 show that there was a 
small reduction of mean decision errors between pre- and post- intervention in the comparison 
group. 

Evaluation Question 13: Relative to pre-implementation, did the IASP process result in a 
reduction of the number of suspect measures in classification of learning disabled students? 

Statistical significance. The Chow test analysis of the effect of the IASP on suspect 
measures is presented in Table 9. The table presents the t-values for the intercept and slopes 
associated with the number of suspect measures in classification of learning disabled students. 
Data show that the reduction of suspect measures between pre- and post-intervention for the 
experimental group was significant, and the change in the trend between pre- and post- 
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intervention was insignificant with slope at the .05 level. A similar reduction in the number of 
suspect measure errors was not noted for the comparison group. 

Educational significance. As shown by the effect sizes in Table 7, there was a statistically 
significant reduction of mean suspect measures between pre- and post-intervention (ES = -1.74; 
p = 0.000) at the probability level of .05 in the experimental group. This result indicates that 
the mean number of errors dropped approximately two standard deviations below mean errors 
prior to intervention, an educationally significant difference. The reduction in mean errors of 
suspect measures between pre- and post-intervention for the comparison group was not significant 
at the .05 level. 

Evaluation Question 1.4: Relative to pre-implementation, did the IASP process result in a 
reduction of the number of procedural errors in classification of learning disabled students? 

Statistical significance. An analysis of the effect of the IASP on procedural errors between 
per- and post- intervention based on the Chow test is presented in Table 9. In the experimental 
group, the reduction of procedural errors between pre- and post-intervention was significant with 
intercept (t = -4.591; p = 0.000) and the change of the trend of procedural errors across time 
was insignificant with slope (t = 0.421; p = 0675) at the .05 level. Both the intercept (t = 
0.421; p = 0.675) and slope (t = -1.862; p = 0.067) were statistically insignificant at the .05 
level for the comparison group. That is, the number of procedural errors failed to decrease or 
increase, and the trend of procedural errors was relatively consistent across time. 

Educational significance. In Table 7, the results indicate that there was a statistically 
significant reduction of mean procedural errors between pre- and post-intervention (ES = -1.58; 
p = 0.000) for the experimental group at the .05 probability level, an educationally significant 
difference. As Table 8 shows, there was not a similar reduction in procedural errors from pre- 
to post-intervention for the comparison group. 

Intellectually Handicapped 

Descriptive Statistics 

Mean overall errors, mean decision errors, and mean of suspect measures and procedural 
errors per file of student classified as intellectually disabled are presented for experimental and 
comparison groups in Figure 12, 13, and 14 respectively. Following intervention, the difference 
of mean overall errors between groups was 2.83 errors per student file, with the experimental 
group averaging 1.92 errors as compared to the comparison group with an average of 4.75 
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IH1 



IH2 



Total 



2 3 4 

Mean Errors per Student (N=2) 

| Prior to Intervention 

IH1: Decision errors 
IH2: Suspect measures 
Procedural errors 

Figure 12 . Mean Errors per IH Student from School 1. 



] After Intervention 



IH1 



IH2 



Total 




2 3 
Mean Errors per Student (N=6) 

Prior to Intervention | 

EM: Decision errors 
IH2: Suspect measures 
Procedural errors 



J After Intervention 



Figure 13 . Mean Errors per IH Student from School 2. 
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IH1: Decision errors 
IH2: Suspect measures 
Procedural errors 



Figure 14 . Mean Errors per IH Student from School 3. 

errors. As noted in Figure 13, there was a significant difference of mean decision errors (0.46 
errors per student file) between experimental (0.54 errors per student file) and comparison (1.00 
errors per student file) groups. Similarly, the mean errors of suspect measures and procedural 
errors of the experimental group (1.31 errors) are significantly less than those of comparison 
group (3.75 errors) (see Figure 14). 

Further analyses of the different types of errors per student files are graphically displayed 
by school in Figure 15, 16, and 17. These figures present the differences of mean errors between 
pre- and post-intervention in the experimental schools. A decrease in errors was reported in all 
four schools. As noted earlier, there were no students who were newly classified as intellectually 
disabled in School 4 during the post-intervention period, therefore, School 4 was not included. 

Mean errors and associated standard deviations of pre- and post-interventions are presented 
for the experimental and comparison group in Table 10 and 11, respectively. Prior to 

t 

intervention, the experimental group averaged 4.81 overall errors per student file. Overall errors 
then dropped to an average of 1.92 during post-intervention, a statistically significant decrease 
of 2.89 errors per student file (see Table 10). As indicated in Table 1 1, the number of overall 
errors per file in the comparison group remained approximately the same, dropping from 5.00 
to 4.75. 



36 



34 



Mean decision errors for the experimental group dropped from 1 .41 to 0.54 per student file 
after intervention, a statistically significant decrease of 0.87 decision errors per student file. 
Again, the number of decision errors remained approximately the same for the comparison group. 



Table 10. 

Pre- and Post- Interventio n Data Analysis for Experimental Group 



Dependent Variables 


Pre-intervention 


Post-intervention 


Effect 


p-value 




Mean 


SD 


Mean 


SD 


Size 




Intellectually Disabled 




Overall Errors 


4.81 


1.52 


1.92 


0.64 


-1.90 


0.000* 


Decision Errors 


1.41 


1.08 


0.54 


0.52 


-0.80 


0.022* 


Suspect Measures with 
data & Procedural Errors 


3.41 


1.01 


1.31 


0.48 


-2.08 


0.000* 



Table 11. 

Pre- and Post- Intervent ion Data Analysis for Comparison Group 



Dependent Variables 


Pre-intervention 


Post-intervention 


Effect 


p-value 




Mean 


SD 


Mean 


SD 


Size 




Intellectually Disabled 




Overall Errors 


5.00 


1.73 


4.75 „ 


1.60 


-0.14 


0.700 


Decision Errors 


1.37 


1.26 


1.00 


0.95 


-0.29 


0.439 


Suspect Measures with 
data & Procedural Errors 


3.63 


0.76 


3.75 


1.14 


0.16 


0.679 



Data in Table 10 show that the number of suspect measures and procedural errors for the 
experimental group significantly decreased to 2.10 errors per student file. As indicated in Table 
11, the number of errors in the comparison group increased from 3.63 (pre) to 3.75 (post). 



Evaluation Question 2.1: Relative to pre-implementation, did the IASP process result in a 
reduction of the number of overall errors in classification of intellectually disabled students? 
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Statistical significance. The Chow test analyses of the effect of die IASP on overall errors 
is presented in Table 12. The effect of the IASP was significant with intercept (t = -2.961; p 
= 0.005), but insignificant with slope (t = 0.059; p = 0.953) at the probability level of 0.05 
for the experimental group. In the comparison group, both intercept (t = 0.204; p = 0.840) and 
slope (t = 0.216; p = 0.831) of the number of overall errors were statistically insignificant at 
the .05 level. Results suggest that while the number of overall errors decreased significantly in 
the experimental group, a corresponding deduction in the overall errors did not occur in the 
comparison group. 

Educational significance. Effect sizes of mean overall errors in the experimental group 
across pre- and post-intervention are included in Table 10. Based on the results, the mean overall 
errors in the experimental group was reduced approximately two standard deviations from pre- 
to post-intervention. A similar reduction was not noted in the comparison group (see Table 11). 



Table 12. 

Analysis of the effect of the IASP on the classification of handicapped students in Ogden School 
District. 



Dependent 
Variables 

Intellectually 
Disabled 


Experimental Group 


Comparison Group | 


Intercept 


Slope 


Intercept 


Slope 


t 


P 


t 


P 


t 


P 


t 


P 


Overall Errors 


-2.961 


0.005* 


0.059 


0.953 


0.204 


0.840 


0.216 


0.831 


Decision Errors 


-1.595 


0.119 


0.723 


0.474 


-0.323 


0.749 


0.632 


0.535 


Suspect Measures 
with Data & 
Procedural Errors 


-2.996 


0.005* 


-0.223 


0.825 


0.780 


0.442 


•0.395 


0.696 



Evaluation Question 2.2: Relative to pre-implementation, did the IASP process result in a 
reduction of the number of decision errors in classification of intellectually disabled students? 

Statistical significance. An analysis of the effect of the IASP on decision errors based on 
the Chow test is presented in Table 12. Results indicate that a reduction of decision errors 
between pre- and post-intervention was insignificant with intercept (t = -1.595; p = 0.119) for 
the experimental group. Further, the change of the trend of decision errors across time between 
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pre- and post-intervention was also insignificant with slope. These findings sharply contrast with 
the previous results. A number of possible explanations exist. 

First, the Chow test is generally conservative (Kmenta, 1986), and although there was a 
decrease in decision errors (see Table 11), the reduction was insufficient in light of the 
conservative requirements of the statistic. Second, there were a number of overriding decisions 
that occurred throughout the consultation (e.g., high adaptive score compared to low IQ, 
borderline IQ or vision and/or hearing problem, etc.). Consequently, it could be said that the 
expert system accounted for a number of potential decision errors in the experimental group 
which do not appear as a part of the data. The number of decision errors per student file in the 
comparison group failed to decrease or increase, and the trend of decision errors was consistent 
across time. 

Educational significance. The analysis of the effect of the IASP between pre- and post-mean 
of decision errors and corresponding effect sizes are presented Table 10. In the experimental 
group, there was a statistically significant reduction of mean decision errors between pre- and 
post-intervention at the 0.05 level. Mean decision errors dropped an average of one standard 
deviation from pre- to post-intervention, an educationally significant finding. As indicated in 
Table 11a similar reduction of mean decision errors did not occur in the comparison group. 

Evaluation Question 2.3: Relative to pre-implementation, did the IASP process result in a 
reduction of the number of suspect errors and procedural errors in classification of intellectually 
disabled students? 

Statistical significance. Table 12 presents the T-values for the intercept and slopes 
associated widi the number of suspect measures and procedural errors in classification of 
intellectually disabled students. The experimental group had a statistically significant reduction 
of suspect measures and procedural errors between pre- and post-intervention, as indicated by the 
intercept. The change in the trend between pre- and post-intervention was insignificant at the .05 
level. In the comparison group, both intercept and slope were statistically insignificant at the 
0.05 level. 

Educational significance. The analysis of the effect of the IASP on the number of suspect 
measures and procedural errors is presented with effect sizes in Table 10. Statistically significant 
effect sizes in the experimental group indicate that the mean numbers of suspect measures and 
procedural errors dropped more than two standard deviations from pre- to post-intervention an 
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educationally significant improvement in teacher's performance in the experimental schools. The 
reduction of suspect measures and procedural errors in the comparison group was negligible. 

IEP Procedures 

Evaluation Question 3.1: Relative to pre-implementation, did the IASP process result in a 
reduction of the number of overall compliance errors with federal and state mandated IEP 
procedures? 

Descriptive Statistics 

Figure 15 presents the mean of total compliance errors (i.e., errors prior to the IEP meeting, 
errors associated with the development of the IEP documents for experimental and comparison 
groups during post-intervention period). Following intervention, the difference between groups 
were a mean of 2.91 compliance errors per student file, with an average of 1.31 errors in the 
experimental group and 4.22 in the comparison group. There was a significant difference of 
mean errors prior to IEP meeting between experimental (0.5 errors) and comparison (1 .44 errors) 
groups. A similar pattern between the experimental and control group performance can be seen 
for mean errors associated with IEP meeting and IEP document. 




Mean Errors per Student 




Exp. Group (N=32) |!iig||| Comp. Group (N=18) 



MCI: Prior to IEP meeting 
MC2: Associated with IEP meeting 
MC3: Development of ffiP documents 

Figure 15. Mean Errors between Experimental and Comparison Group. 
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Further analyses of the different types of errors per student files are graphically displayed 
by school in the experimental group in Figures 16, 17, 18, and 19. Data indicate that schools 
in the experimental group reduced the number of compliance errors by half from pre- to post- 
intervention. 




Prior to Intervention 



2 3 
Mean Errors per Student (N=10) 

t 



M'-: | After Intervention 



MCI: Prior to IEP meeting 

MC2: Associated with IEP meeting 

MC3: Development of IEP documents 



Figure 16 . Mean Errors per Student from School 1. 



MCI 



MC2 



MC3 



Total 




Prior to Intervention 



2 3 
Mean Errors per Student (N=5) 

■ 



] After Intervention 



MCI: Prior to IEP meeting 
MC2: Associated with IEP meeting 
MC3: Development of IEP documents 



Figure 17 . Mean Errors per Student from School 2. 
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Prior to Intervention 



2 3 
Mean Errors per Student (N=7) 

: ".;v 1 After Intervention 



MCI: Prior to IEP meeting 
MC2: Associated with IEP meeting 
MC3: Development of IEP documents 



Figure 18 . Mean Errors per Student from School 3. 



MCI 
MC2 
MC3 
Total 



12.44 



1.44 




0 



2 3 4 5 

Mean Errors per Student (N=10) 

I After Intervention 



Prior to Intervention 



MCI: Prior to IEP meeting 

MC2: Associated with IEP meeting 

MC3: Development of IEP documents 

Figure 19 . Mean Errors per Student from School 4. 

Mean errors and associated standard deviations for pre- and post-intervention are presented 

for experimental and comparison group in Tables 13 and 14 respectively. Prior to the 

intervention, the experimental group averaged 4.26 overall compliance errors per student file. 
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Table 13. 

Pre- and Post- Intervention Data Analysis for Experimental Group 



Dependent Variables 


Pre-intervention 


Post-intervention 


Effect 


p-value 




Mean 


SD 


Mean 


SD 


Size 




Mandate Consultant 




Overall Errors 


4.26 


1.36 


1.31 


0.78 


-2.16 


0.000* 


Problems prior to IEP 
meeting 


2.12 


0.83 


0.50 


0.51 


-1.95 


0.000* 


Problems associated with 
IEP meeting 


0.95 


0.54 


0.13 


0.34 


-1.54 


0.000* 


Problems associated with 
the development of IEP 
documents 


1.36 


0.58 


0.63 


0.49 


-1.27 


0.000* 



Table 14. 

Pre- and Post- Intervention Data Analysis for Comparison Group 



Dependent Variables 


Pre-intervention 


Post-intervention 


Effect 
Size 


p-value 


Mean 


SD 


Mean 


SD 


Mandate Consultant 


Overall Errors 


4.27 


1.25 


4.22 


1.21 


-0.04 


0.892 


Problems prior to the IEP 
meeting 


1.61 


0.91 


1.44 


0.78 


-0.18 


0.534 


Problems associated with 
the IEP meeting 


0.73* 


0.63 


0.72 


0.57 


-O.01 


0.957 


Problems associated with 
the development of IEP 
documents 


1.98 


0.65 


2.06 


0.53 


0.12 


0.670 



The overall errors then dropped to an average of 1.31 per student file during the post- 
intervention, a statistically significant decrease of 2.95 errors. Performance of the comparison 
group remained the same from pre- to post-intervention. 
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In the experimental group, the mean number of errors prior to the IEP meeting were reduced 
from 2.12 errors (pre) to .05 (post), a statistically significant decrease of 1.61 errors (see Table 
13). Performance of the comparison group remained unchanged (see Table 14). 

The mean number of errors associated with the development of the IEP document for the 
experimental group dropped from an average of 1.36 errors (pre) to 0.63 (post) (see Table 13). 
As indicated in Table 14, the average number of errors in the comparison group increased 
slightly from 1.98 errors (pre) to 2.06 errors (post) per student file. 

Finally, Table 13 shows that the number of errors associated with IEP meetings for the 
experimental group dropped from 0.95 errors to 0.13 errors per student file, a significant 
decrease of 0.82. The number of errors in the comparison group remained relatively constant. 

Statistical significance. The Chow test analyses of the effect of the IASP on overall 
compliance errors is presented in Table 15. As Table 15 shows, the effect of the IASP was 
significant with intercept (t = -3.744; p - 0.004), but insignificant with slope (t = 0.207; p = 
0.836) at the .05 level for the experimental group. In the comparison group, the reduction in 
overall compliance errors was statistically insignificant at the .05 level. 
Table 15. 

Analysis of the effect of the IASP on the classification of handicapped students in Ogden School 
District. 



Dependent 
Variables 

Mandate Consultant 


Experimental Group 


Comparison Group 


Intercept 


Slope 


Intercept 


Slope 


t 


P 


t 


P 


t 


P 


t 


P 


Overall Errors 


-3.744 


0.004* 


0.207 


0.836 


0.515 


0.608 


-0.314 


0.754 


Problems prior to 
the IEP meeting 


-2.915 


0.005* 


0.612 


0.542 


1.179 


0.243 


1.133 


0.262 


Problems associated 
with IEP meeting 


-2.011 


0.048* 


0.014 


0.989 


-0.124 


0.902 


-1.094 


0.279 


Problems associated 
| with development of 
1 IEP documents 


-2.993 


0.004* 


-1.025 


0.309 


-0.138 


0.891 


-0.912 


0.366 



Educational significance. The Effect Sizes (ES) between pre- and post-mean overall errors 
in the experimental group are displayed in Table 13. There was a statistically significant 
reduction of overall IEP compliance errors (ES = -2.16; p = 0.000) at the .05 level with errors 
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dropping an average of more than two standard deviations from pre- to post-intervention. As 
Table 14 indicates, the reduction of overall errors for the comparison group was negligible (ES 
= -0.04; p = 0.892). 

Evaluation Question 3.2: Relative to pre-implementation, did the IASP process result in a 
reduction of the number of errors prior to the IEP meeting with federal and state mandated IEP 
procedures? 

Statistical significance. Table IS presents the t-values for the intercept and slopes associated 
with the number of IEP procedural errors prior to the IEP meeting. In the experimental group, 
the reduction of errors prior to the IEP meeting between pre- and post-intervention was 
significant with intercept. However, the change of the trend of errors across time (pre- and post- 
intervention) was insignificant with slope at the .05 level. In the comparison group, both 
intercept (t - 1 . 179; p = 0.243) and slope (t = 1 . 133; p = 0.262) were statistically insignificant. 

Educational significance. The analysis of the effect of the IASP on errors prior to the IEP 
meeting is presented with effect sizes in Table 13. Effect sizes for the experimental group were 
statistically significant between pre- and post-intervention (ES = -1.95; p = 0.000) at the .05 
level. In the experimental group, mean procedural errors prior to the IEP meeting dropped an 
average of almost two standard deviations from pre- to post-intervention, a substantial decrease 
at the (0.000) level. As Table 14 shows, a similar reduction of errors between pre- and post- 
intervention was not noted in the comparison group (ES = -0. 18; p = 0.534). 

Evaluation Question 3.3: Relative to pre-implementation, did the IASP process result in a 
reduction of the number of errors associated with the development of the IEP document with 
federal and state mandated IEP procedures? 

Statistical significance. The Chow test analyses of the effect of the IASP are presented in 
Table 15. The data in Table 15 reveals a similar pattern to the previous data. The number of 
errors associated with the IEP document in the experimental group were greatly reduced from 
pre to post-intervention although the change in the trend was insignificant with slope at the .05 
level. In the comparison group, reduction of the number of errors associated with the IEP 
documents were statistically insignificant at the .05 level. 

Educational significance. As shown in Table 13, effect sizes in the experimental group 
were statistically significant between pre- and post-intervention (ES = -1.54; p = 0.000) at the 
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.05 level. Results for the comparison group were not statistically significant (ES = -0.01; p = 
0.957) (see Table 14). 

Evaluation Question 3.4: Relative to pre-implementation, did the IASP process result in a 
reduction of the number of errors associated with the mandated procedures in IEP meeting. 

Statistical significance. An analysis of the effect of the IASP based on the Chow test is 
presented in Table 15. The findings in Table 15 are consistent with the previous data. The 
reduction of errors associated with the IEP meeting between pre- and post-intervention was 
significant at the .05 level for the experimental group but not for the comparison group. 

Educational significance. Effect sizes in the experimental group indicate that there was a 
statistically significant reduction of mean errors associated with the IEP meeting (ES = -1.27; 
p - 0.000), (see Table 13). A similar reduction of mean errors was not noted in the comparison 
group (ES = 0.12; p = 0.67). 
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S 

SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS 

Research shows that many educators continue to have difficulty (1) interpreting and applying 
special education rules and regulations (2) determining eligibility for special education, and (3) 
developing IEPs. Such problems may prove costly-children in need of special education may 
fail to receive the services they require, other children not requiring services may be 
inappropriately classified and served, thus wasting precious special education resources. Clearly, 
school administrators, teachers, parents', and children would benefit if there were an objective, 
knowledgeable expert in the field to assure that federal and state mandated procedures were being 
implement* d as intended. 

In recent years, expert systems have been developed in the areas of medicine, geology, and 
engineering as a means of capturing specific domains of human expertise and making that 
knowledge available to inexperienced individuals. Most recently, researchers at Utah State 
University have successfully applied the technology of expert systems to the area of federal 
regulations and state rules for special education. These researchers developed and validated 
several expert systems including (1) Class.LD2, (2) Class.IH, (3) Class.BD, and (4) Mandate 
Consultant. Results of validation studies and replications have been consistent with the findings 
of other special education researchers (Colbourn & McLeod, 1983; Haynes, J. A., Pilato, V. H., 
& Malouf, D. B., 1987). Together, these studies support the following conclusions; 

1 . Expert systems can be developed to substantially enhance decision-making in special 
education. 

2. Based on research conducted to date, expert systems technology appears well-suited to 
decision-making in special education. 

3. Expert systems can serve multiple functions (e.g., decision maker, inservice trainer) 
which considerably increases their cost-effectiveness. 

Given the well-documented interest in computerized, information management systems, the 
demonstrated effectiveness of the expert systems to enhance decision making, and the present 
investment in several expert systems, the development of a comprehensive information 
management system was feasible and necessary. 

The primary purpose of the present study was the development and initial validation of a 
computerized information system that used expert system technology. To meet this purpose, this 
Intelligent Administrative Support Program (IASP) incorporated existing expert systems (i.e., 
Class. LD2, Class.IH, Mandate Consultant) with a central special education data base. 
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Specifically, the program was designed to assist special education administrators in the acquisition 
and management of information regarding on-going staff training needs. 

The development of the computerized management system followed a Research and 
Development (R&D) model consisting of four phases. The first three phases emphasized 
formative evaluation. They included: 

1 i Content clarification and system design: During this phase, a group of professionals met 
on several occasions to determine the objectives for the development of the system and 
to clarify who would use the program, how the program would be used, and what type 
of output would be most useful in evaluating staff needs. 
2. Development and field test of the prototype and revisions: In this phase, several 
prototypes were developed and then tested and revised until it produced reliable and 
functional output. Several meetings were held with district staff to review output results 
and to revise the format of the system's summary report to facilitate understanding. 
The third phase of the R & D model involved a summative evaluation of the computerized 
information management system. An interrupted multiple time-series design (Glass, Wilson, & 
Gottman, 197S) emplcyi ig a non-equivalent comparison group, was used to investigate the effect 
of the IASP process on district compliance problems in the assessment and placement procedures 
used in the classification of students for special education. Failure to implement federal and state 
regulatory procedures for IEP development were also evaluated. The types of errors examined 
were: overall errors, decisions errors, use of suspect measures, procedural errors, and the degree 
to which student's individual educaiton plans (IEP) complied with state and federal regulations. 
A Chow-test was the statistical procedure used to determine if there was a significant difference 
between regressions lines (number of errors and/or change in trend) during pre- and post- 
intervention assessment. 

To assess potential compliance problems, two data collectors evaluated a total of 383 files 
of students who had been classified as learning disabled (LD) or intellectually handicapped (IH). 
Student files were selected from four elementary schools (experimental group) and three 
secondary schools (comparison group). Files were reviewed prior to and following district 
interventions, using three validated expert systems (Class.LD2, Class.IH, Mandate Consultant). 
Based on the IASP findings, the district developed staff interventions that ranged from large 
group inservices to one-on-one. IASP follow-up data indicated that the interventions were 
ineffective; at which point, the district totally revised their staff development procedures and 
implemented a "mentor team" to monitor the classification and placement procedures for all 
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students referred for admission to special education. The mentor team used the three expert 
systems (Class.LD2, Class .IH, Mandata Consultant) to review all requisite data with the special 
education teacher. The district used IASP monitoring and the mentor team approach throughout 
the 1990-1991 school year. 

The summative evaluation yielded the following findings: 

Use of the IASP data-based management, the expert systems, and the district intervention 
(i.e., use of mentor teachers) resulted in a dramatic decrease in the reduction of compliance 
errors in student files in the areas of LD and IH classification, and IEP development from 
pre- to post-assessment for schools in the experimental group. Performance of the 
comparison group remained unchanged as measured during pre- and post-intervention. 
Further analyses showed that the reduction in errors for the experimental group was 
statistically and educationally significant. There were no educationally significant findings 
for the comparison group. 

Further, anecdotal information suggests that administrators and staff were pleased with 
the information provided by the IASP process and felt that it was extremely helpful in 
designing effective intervention strategies, and in providing current and relevant data 
regarding specific staff development. It whould be noted that based on the success of the 
IASP process, the district intends to expand the program into its secondary schools for the 
upcoming school year. 

These findings strongly suggest that the combination of expert systems technology and a 
special education administrative data base created a powerful support system for progressively 
improving the quality of decisions made by multidisciplinary teams. Such decisions can have life- 
long implications for students begin considered for special education. 

Based on these results, the following conclusions can be drawn. 

1 . The technology of expert systems and a central administrative data base can be combined 
to create an effective decision support system that can provide continuous feedback to 
administrators regarding the needs and on-going development of their staff. 

2. The IASP can provide special education administrators with current information 
regarding the quality of classification decisions and regulatory procedures pertaining to 
IEP development. To increase its flexibility and value, the IASP can offer a variety of 
data options to the administrator. By cross-referencing with individual staff, and special 
education classification units, the administrator can easily pinpoint strengths and 
weaknesses on a general or individual basis in their district. 
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3. Because IASP was designed to integrate data bases from a variety of expert systems, the 
IASP can be easily upgraded to include the most relevant and valid systems. If a new 
expert system is developed, the addition of a data base from that system would be far 
less expensive than the development of a sophisticated front-end data base program. This 
design flexibility enhances the cost-effectiveness of the technology. 

Limitations 

Although the results of the present project are promising, it is important to note limitations 
of the data. The first limitation concerns the use of a time-series analysis that assumes equal time 
intervals in the assessment process. Due to the nature of the data, equal intervals across time 
were not possible as the majority of special education students are classified at the beginning of 
the school-year, a small number at the end of the school year, and relatively few in between. 
In this study, the files analyzed by the IASP process were sorted by the student's evaluation time, 
and corresponding outputs were then plotted on the x-axis according to their chronological order. 
Therefore, the results of the trend analysis (e.g., change in progressive increasing or decreasing 
errors by time) should be interpreted with the time limitation in mind. 

A second limitation involves the sample population. In this study, elementary schools were 
used as an experimental group and secondary schools as a comparison group. Elementary schools 
were not selected to serve in the control group due to potential problems with contamination 
discussed in the Procedure Section. Because elementary and secondary schools may vary across 
a number of variables (e.g. service pattern, varying instructors, etc.), results of the study may 
only generalize to elementary schools. 

Implications 

Based on the findings, there were several implications, First, the review of literature clearly 
documented the need for effective information management system for special education 
administrators. Evidence obtained in this project suggests that special education administrators 
can effectively use feedback in the form of district-wide staff performance data to formulate and 
implement effective staff development programs. 

Second, results of this study suggest that the IASP and previously developed systems have 
the potential to support staff development in two ways. First, conclusions were available in a 
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prescriptive manner to direct staff development (e.g., identify which staff required training in 
what content areas). Additionally, on-going information served to evaluate the impact of the staff 
development and training on current performance. 

The expert systems can act as effective preservice and inservice training tools. These systems 
can be used in their existing form or adapted to provide training models of appropriate decision- 
making. For example, in this study, members of the mentor team assisted teachers with the 
assessment, classification, and placement process. Teachers were walked step-by-step through 
the process beginning with the referral and ending with development of appropriate goals and 
objectives for an IEP. 

Finally, although the IASP was designed to identify areas of compliance according to federal 
and Utah State rules, this system could easily be adapted for use by other states. In addition, 
because the field-test was conducted in a school district with the highest percentage of minority 
students in the state, the lowest tax base, and one of the highest student-to-teacher ratios in the 
nation, it is expected that the combination of expert systems technology and the IASP process 
could have beneficial results for other states with similar population characteristics. 

Recommendations 

The results of this study lead to the following recommendations: 

1 . In the present study, the summative field-test was conducted in a site with one of the 
highest minority student populations in the state. Current expert systems need to be 
expanded to ask more detailed information regarding disclaimer issues such as culture. 
Class. LD2 currently addresses cultural issues in a cursory fashion. Questions pertaining 
to a student's cultural background cover such areas as bilingualism, proficiency of 
English, number of years living in the United States, and percent of minorities in the 
student's school. Additional information regarding performance on formal bilingual tests 
and whether cultural problems preclude special education classification need to be 
addressed in all expert systems relating to classification. 

2. To further validate the effectiveness of the IAS? process, a replication of this study is 
needed. Additional research using secondary schools, and additional expert systems 
(e.g., Class.BD, Class.PH, or Class.CD) in conjunction with the IASP should be 
conducted. One important dependent variable that should be included in a replication 
should be the number of students classified in special education. While the present study 
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examined the reduction of compliance errors as a result of the IASP process, it was not 
clear whether there was a corresponding reduction in the number of students classified 
in special education. 

Finally, the development and validation of the IASP process should be considered a 
beginning. Future studies should explore the variety of potential applications for 
computerized information management systems for special education administrators. 
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SAMPLE CONSTRAINTS: 

This sample is drawn from class. in records. Intellectual handicap issues are the primary 



focus of the class. ih expert system. 

AH class. ih caches were included in this report. 

Report on 
Problems with Data 

School: A 

□ An academic test was not age-appropriate 1 

> 

Student 1 

□ Missing hearing evaluation 1 

Student 2 

□ Missing speech/language evaluation 1 

Student 3 

□ The adaptive test was not age-appropriate 5 

Student 4 
Student 5 
Student 6 
Student 7 
Student 8 

□ Widely differing IQ and adaptive scores have been reported 4 

Student 9 
Student 10 
Student 11 
Student 12 

Total number of values 12 

School B 

P Missing hearing evaluation 2 

Student 1 
Student 2 

□ Missing vision evaluation 7 



Student 3 
Student 4 
Student 5 
Student 6 
Student 7 
Student 8 
Student 9 
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□ Old test data was used , 1 

Student 8 

□ The adaptive test was not age-appropriate ; 1 

Student 9 

□ Widely differing IQ and adaptive scores have been reported 5 

Student 10 
Student 11 
Student 12 
Student 13 
Student 14 

Total number of Values 16 

School: C 

□ An academic test was not age-appropriate 1 

Student 1 

□ Missing hearing evaluation 1 

Student 2 

□ Missing vision evalution 2 

Student 3 
Student 2 

□ Old test data was used 3 

Student 4 
Student 5 
Student 6 

□ Widely differing IQ and adaptive scores have been reported 2 

Student 3 
Student 7 

Total number of Values 9 

School: D 

□ Missing hearing evaluation 2 

Student 1 
Student 2 

□ Missing vision evalution 2 

Student 1 
Student 2 

Total number of values 4 
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School: E 

□ An academic test was not age-appropriate 1 

Student 1 

□ Widely differing IQ and adaptive scores have been reported 1 

Student 1 

Total number of values 2 
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Report on 
Procedural Problems 

The data for this report is found in 7 student fiels from 4 schools. Value(s) are reported in 
a school by school format. 

School: A 

□ There was more than 30-days between parental permission & assessment dates 1 



Student 1 

Total number of values 1 

School: B 

□ There was more than 30-day between parental permission & assessment dates . 1 
Student 1 

Total number of values !□ 



School: C 

□ There was more than 30-day between parental permission & assessment dates . 4 

Student 1 
Student 2 
Student 3 
Student 4 



Total number of values 4 

School: D 

□ There was more than 30-day between parental permission & assessment dates . 1 
Student 1 

Total number of values ID 

Report on 
Non-Hf Explanations for the Problem(s) 



The data for this report is found in 5 student files from 1 school. Value(s) are reported for 



the entire described sample as a group. 
School: A 

□ Behavioral problems might account for the classroom difficulty 1 

Student 1 

□ Cultural problems might account for the classroom difficulty 3 

Student 1 
Student 2 
Student 3 

Total number of values 4 
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Sample Constraints: 

This sample is drawn from class.ld records. Learning disability issues are the primary focus 
of the Class.LD Expert System. 

Limiting factor: School name 

Limiting Value(s): Students with a value of 'TOSmith' were included in the sample 

Report on 
Overall System Advice 



Bahavioral problems 1 

School: 'TOSmith' 
Student 1 

Intellectually handicapped 1 

School: 'TOSmith' 
Student 2 

Learning disabled 8 



School: 'TOSmith' 
Student 3 
Student 4 
Student 5 
Student 6 
Student 7 
Student 8 
Student 9 
Student 10 



Vision problems 2 

School: 'TOSmith' 

Student 5 

Student 10 

Total Number of Values 12 



SAMPLE CONSTRAINTS: 

This sample is drawn from class.ld records. Learning disability issues are the primary focus 
of the class.ld expert system. 

Limiting Factor: School Name. 

Limiting Value(s): Students with a value of 'TOSmith' were included in the sample. 

Report on - 
Problems with Data 

School: 'TOSmith' 
Name: Student 1 
Missing hearing evalution of 100 
Name: Student 2 
The hearing test data was missing of 100 
Name: Student 3 
Hearing data was old of 100 
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Report on 
Procedural problems 



The data for this report is found in 2 student files from 1 school. Value(s) are reported in 
a student by student format. 

School: 'TOSmith' 
Name: Student 1 

Two or more QUALITY prereferral interventions have NOT been tried of 23 
Name: Student 2 

Two or more QUALITY prereferral interventions have NOT been tried of 32 



SAMPLE CONSTRAINTS: 



This sample is drawn from class.ld records. Learning disability issues are the primary focus 
of the class.ld expert system. 

Limiting Factor: School name. 

Limiting Value(s): Students with a value of "TOSmith' were included in the sample. 

Report on 
Overall System Advice 

The data for this report is found in 5 student files from 1 school. Value(s) are reported for 
the entire described sample as a group. 

Learning disabled 5 

Student 1 
Student 2 
Student 3 
Student 4 
Student 5 



Visual problems 2 

Student 2 
Student 5 

Total Number of Values 7 

Report on 

Non-LD Explanations for the Problem(s) 

The data for this report is found in 1 student file from 1 school. Value(s) are reported in a 
student-by-student format. 



School: 'TOSmith' 
Name: Student 5 

Vision problems might account for the classroom difficulty of 100 
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SAMPLE CONSTRAINTS: 



This sample is drawn from mandate consultant records. IEP Compliance issues are the 
primary focus of the mandate consultant expert system. 

Limiting Factor: The student's School's name. 

Limiting Value(s): Students with a value of 'TOSmith* were included in the sample. 



Report on 

Problems Associated with School Activities 
Prior to the IEP Meeting 

The data for this report is found in 8 student files from 1 school. Value(s) are reported in 
a student-by-student format. 

School: 'TOSmith* 
Name: Student 1 

The student's parents were not told who the IEP team participants would be of 100 
Two or more quality interventions were NOT tried of 100 

Name: Student 2 

After evalution, it took more than 30 days to develop the IEP of 100 

Fewer than 5 working days notice were given to parents prior to the IEP meeting of 100. 

There were more than 30 days between permission to evaluate and the evaluation of 100. 

Name: Student 3 

Fewer than 5 working day notice were given to parents prior to the IEP meeting of 100. 
There were more than 30 days between permission to evaluate and the evaluation of 100. 

Name: Student 4 

Fewer than 5 working days notice were given to parents prior to the IEP meeting of 100. 
Name: Student 5 

After evaluation, it took more than 30 days to develop the IEP of 100. 

Fewer than 5 working days notice were given to parents prior to the IEP meeting of 100. 

Two or more QUALITY interventions were NOT tried of 100. 

Name: Student 6 

Fewer than 5 working days notice were given to parents prior to the IEP meeting of 100. 
Name: Student 7 

After evaluation, it took more than 30 days to develop the IEP of 100. 

Fewer than 5 working days notice were given to parents prior to the IEP meeting. 

Name: Student 8 

Fewer than 5 working days notice were given to parents prior to the IEP meeting of 100. 
There were more than 30 days between permission to evaluate and the evaluation of 100. 
Two or more quality interventions were NOT tried of 100. 
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Report on 

Problems Associated with the IEP Meeting 

The data for this report is found in 5 student files from 1 school. Value(s) are reported in 
a student-by-student format. 

School: 'TOSmiuY 

Name: Student 1 

No representative of the public agency attended the IEP meeting of 100. 
No surrogate parent attended the meeting and one was clearly needed of 100. 

Name: Student 2 

The student should have attended the IEP meeting but he/she did not attend of 100. 
Name: Student 3 

No representative of the public agency attended the IEP meeting of 100. 
Name: Student 4 

The student should have attended the IEP meeting but he/she did not attend of 100. 
Name: Student 5 

The student should have attended the IEP meeting but he/she did not attend of 100. 

Report on 

Problems Associated with the Development 
of the IEP Document 

The data for this report is found in 6 student files from 1 shcool. Value(s) are reported in 
a student-by-student format. 

School: 'TOSmiuY 

Name: Student 1 

The percentage of time in regular education is not listed on the IEP of 100. 
Name: Student 2 

NOT all of the objectives had clearly stated outcomes of 100. 

The percentage of time in regular education is not listed on the IEP of 100. 

Name: Student 3 

NOT all of the objectives had clearly stated outcomes of 100. 

The percentage of time in regular education is not listed on the IEP of 100. 

Name: Student 4 

The percentage of time in regular education is not listed on the IEP of 100. 



Name: Student 5 

NOT all of the objectives had clearly stated outcomes of 100. 

The percentage of time in regular education is not listed on the IEP of 100. 

Name: Student 6 

NOT all of the objectives had clearly stated outcomes of 100. 

The percentage of time in regular education is not listed on the IEP of 100. 
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Mini audits 
by special 
education 
coordinators 
for 

continued 
compliance 
(spot I 
checks). | 


Results 


Copies of IEPs 
are submitted for 
compliance 
review. Then 
returned with 
comments to the 
teachers - some 
one-on-one 
inservice also. 


How 
long? 


3 hrs 


Who is 
respon- 
sible? 


IEP 

Task 

Force 


Intervention^) 


Description 


An in-depth description 
of compliance in all of 
the target areas was 
given. 


Type 


Inserv. 


G/Ind 


Group 


Target Area(s) 


IEP - See list 
below 


Date 


3/1/90 
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CLASSIFICATION TRACKING RECORD 



SCHOOL 


STUDENT 


CLASSROOM TEACHER 


DATE OF BIRTH 


INITIAL/3 YR. EVALUATION 


GRADE 





CONTACT PERSON: 


DATE: 


PRIOR NOTICE 


REFERRAL ISSUES 


PARENT RIGHTS 
REFERRAL COMPLETED 


PREREFERRAL INTERVENTIONS: 
1. 


PERMISSION TO EVALUATE 


2. 


SUSPECTED HANDICAP 


VISION: DATE RESULTS 
HEARING: DATE RESULTS 


ADDITIONAL PERTINENT INFORMATION: 



INTELLECTUAL FUNCTIONING (I.Q.) ADAPTIVE/SOC. BEHAVIOR m. bd> 

TEST(3) USED: MEASURE USED: 

DATE(S) GIVEN: DATE: 

RESULTS: RESULTS: 



ACADEMIC FUNCTIONING <u. u^. M «m iu^o 








TEST(S) USED: 

DATE: 

RESULTS: 


TEST(S) USED: 

DATE: 

RESULTS: 


TEST(S) USED: 

DATE: 

RESULTS: 
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OBSERVATION^) (1 Requited for LD, 3 Required for initial BD, with peer comptrison) 
#1 DATE: LOCATION: BY WHOM: 

Wl DATE: LOCATION: *" BY WHOM: 

#3 DATE: LOCATION: BY WHOM: 



CLASSROOM DISCIPLINE 



1. 


POSTED RULES 


YES 


NO 


N/A 


2. 


POSTED CONSEQUENCES FOR FOLLOWING RULES 


YES 


NO 


N/A 


3. 


POSTED CONSEQUENCES FOR NOT FOLLOWING RULES 


YES 


NO 


N/A 


4. 


EVIDENCE THAT RULES AND CONSEQUENCES ARE BEING USED 


YES 


NO 


N/A 



COMMENTS: 



PHYSICAL OR HEALTH 


YES 


NO 


EMOTIONAL DISTURBANCE 


YES 


NO 


COMMUNICATIVE 


YES 


NO 


ECONOMIC FACTORS 


YES 


NO 


ENVIRONMENTAL FACTORS 


YES 


NO 


CULTURAL FACTORS 


YES 


NO 


ENGLISH PROFICIENCY 






YEARS RESIDENCY 


ETHNICITY 


%ETHNIC ORIGIN 





RESULTS (Check etch area for which advise was given) DATE OPINION GENERATED 



✓ 


ADVISE 


c.f. 


SUMMARY 




LD 








BD 








IH 








SL 








DNQ 
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CLASS.IH CONSULTATION REPORT 



Consultation Date: 10/20/91 

Student's Name: John Doe 

School: Plymouth 

Age: 8 years 1 month 

APTITUDE INFORMATION: 

Test: Wechester Intelligence Scale for Children - Revised (WISC-R full scale) 

Date: 9/13/91 

Score Name: Full Scale IQ score (standard scores mean = 100 sd - 15) 

Score: 69 

Z Score: -2.06667 

ADAPTIVE INFORMATION: 

Test: Vineland Adaptive Behavior Scales 

Date: 10/25/91 

Score Name: Standard scores mean = 100 sd - 15 

Score: 65 

Z Score: -2.33333 

Basic Reading Skills: 

Test: Woodcock Johnson Psycho-Educational Battery: Part II 

Date: 10-5-91 

Score Name: Achievement standard score for Reading Cluster (mean = 100 SD = 15) 

Score: 62 

Z Score: -2.5333 



ALTERNATIVE SPECIAL EDUCATION: 

No alternative special education problems were reported or those reported were not found to be severe 
enough to be the cause of the student's learning problem. 

NON-SPECIAL EDUCATION OPTIONS: 

Slow learner 

If the information that you have provided is correct, the student probably should not be classified as 
intellectually handicapped. Test data suggest that this is a borderline student who, while being 
technically eligible for special education probably could be best served in a good regular education 
program for children at risk. (CS - 34/100) 

BAD OR MISSING DATA: 

Missing hearing evaluation 
Missing vision evaluation 

FINAL IH PLACEMENT ADVICE: 

If one assumes that the bad or missing data DID NOT substantially mislead the computer program, you 
can be certain that an IH classification is appropriate with a certainty of 99 percent. 
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CLASS.LD CONSULTATION REPORT 

Consultation Date: 10/20/91 

Student's Name: Jane Smith 

Grade: 2.4 

Age: 7 years 1 month 

APTITUDE INFORMATION: 

Test: Wechester Intelligence Scale for Children - Revised (WISC-R full scale) 

Score Name: Full Scale IQ score (standard scores mean = 100 sd - 15) 

Score: 98 

DISCREPANCY INFORMATION: 

Academic Problem Area: Basic reading skills 

Test: Woodcock-Johnson Psycho-Educational Battery: Part II 

Score Name: Achievement standard score for the reading cluster (mean = 100 sd = IS) 

Score: 70 

Based on this information there is a 98 percent likelihood that the discrepancy in basic reading skills 
is severe enough for an LD placement in Utah. 

ALTERNATIVE SPECIAL EDUCATION: 

No alternative special education problems were reported or those reported were not found to be severe 
enough to be the cause of the student's learning problem. 

NON-SPECIAL EDUCATION OPTIONS: 

No alternative education options were found. 
PRECLUSIONS FOR LD PLACEMENT: 

No preclusions for LD placement were found. 
BAD OR MISSING DATA: 

No bad or missing data indicated. 
FINAL LD PLACEMENT ADVICE: 

You can be 98 percent certain that an LD classification is appropriate. 



er|c 



81 



Appendix £ 

Corresponding Graphs Related to Chow Statistics 
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Casewise Plot of Standardized Residual 
*: Selected M: Missing 
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Equation Number l Dependent Variable. ERRNUM 



Casewise plot of Standardized Residual 
*: Selected M: Missing 

-3.0 0.0 3.0 



Case # 


O: . . . . 




ERRNUM 


JET d[\UW 




1 




. * 


6 






2 




. * 


v> 


5 . 91 Oil 

*J * Cm A-\J t 


7896 


3 


* 




3 


5 177K 


—0 1775 

Cm . X / / *J 


4 




* 


7 


5 1 1A5 
•J* J. tl^ 


1 8555 

_L c U J J J 


5 




. * 


v> 


5 1111? 
->> * ill J 


QQQR 
■ O J 


6 




* . 


A 
** 




—i msfi 


7 




* . 


A 

•* 


*J fUH JO 


— 1 ("1A56 


8 




. * 


6 

v> 


5 m op. 


9874 


9 




. * 


7 


4 9796 


2 0204 

Oaf • V £t \J *W 


10 




* . 


4 


4 9467 


— 9467 


11 




* 


5 


4 .9137 


. 0863 


12 




* 


6 


4 8807 


1 1193 


13 




* 


5 


*s • o *s / o 


.1522 


14 




* 


5 


4 .8148 


. 1852 


15 




* . 


4 


4 . 7818 


-.7818 


16 




* 


3 


4 . 7489 


-1.7489 


17 


* 




2 


4 .7159 


-2. 7159 


18 




* 


6 


4 . 6829 


1.3171 


19 


* 




2 


4.6500 


-2 . 6500 


20 


* 




2 


4 . 6170 


-2 . 6170 


21 




. * 


6 


4.5840 


1.4160 


22 


m 


. * 


5 


4.5511 


.4489 


23 




* . 


4 


4.5181 


-.5181 


24 


m 


* 


7 


4 .4851 


2.5149 


25 




. * 


5 


4 .4522 


.5478 


26 




* . 


4 


4 .4192 


-.4192 


27 


• 


* 


6 


4.3862 


1.6138 


28 




* 


1 


2 . 0879 


-1.0879 


29 




. * 


3 


1.9560 


1.0440 


30 




* 


2 


1.8571 


.1429 


31 




* 


2 


1.8242 


.1758 


32 




.* 


2 


1.7582 


.2418 


33 




* 


2 


2.0549 


-.0549 


34 




* . 


1 


2.0220 


-1.0220 


35 




* 


2 


1.9890 


.0110 


36 




. * 


3 


2.1209 


.8791 


37 




* 


2 


1.9231 


.0769 


38 




* 


2 


1.8901 


.1099 


39 




* 


2 


1.7912 


.2088 


40 




* . 


1 


1.7253 


-.7253 


Case # 


-3.0 


0.0 3.0 


ERRNUM 


*PRED 


*RESID 



o 

ERIC 



84 



Equation Number 1 Dependent Variable.. ERRNUM 



Casewise Plot of Standardized Residual 
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Casewise Plot of Standardized Residual 
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Residuals Statistics: 
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Casewise Plot of Standardized Residual 



*: Selected M: Missing- 
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Residuals Statistics: 
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Standardized Scatterplot 
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Casewise Plot of Standardized Residual 
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Equation Number 1 Dependent Variable.. ERRNUM 



Residuals Statistics: 





Min 


Max 


Mean 


*PRED 


-.0082 


5.1938 


3.5045 


*RESID 


-5.0555 


8.2905 


-.0000 


*ZPRED 


-1.7899 


.8608 


.0000 


*ZRESID 


-2.2401 


3.6736 


-.0000 


Total Cases = 


111 







Std Dev 

1.9626 
2.2362 
1.0000 
.9909 



N 

111 
111 
111 
111 
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Standardized Scattcrplot 

Across - *PRED Down - *RESID 

Out EEDDDDDEDDDDDEDDDDDEDDDDDEDDDDDEDDDDDEE 

3 E e Symbols: 

3 3 

3 3 Max N 

2 E : E 

3 .. 3.1. 

3 ... 3:2. 

IE : ... E * 5. 

3 ** . 3 

3 ** ** 1 3 

. 0 E * : e 

3 : . : * 3 

3 : *. .: 3 

Dl E : : * : g 

3 ... 3 

3 3 

D2 E . E 

3 3 

3 3 

D3 E E 
Out EEDDDDDEDDDDDEDDDDDEDDD . DDDDDEDDDDDEE 

D3 D2 Dl 0 1 2.3 Out 



Standardized Scatterplot 

Across - TIMEMC Down - *RES1D 

Out EEDDDDDEDDDDDEDDDDDEDDDDDEDDDDDEDDDDDEE 



3 E E Symbols: 

3 3 

3 3 Max N 

2 E E 

3 : 3.1 

3 ... 3:2 

IE . . . . . E * 4 

3 . *. . . 3 

3 . : . : . *.* 3 

0 E .. .:• E 

3 .:: ... 3 

3 ... . : . . . 3 

Dl E ... E 

3 ... 3 

3 3 

D2 E . E 

3 3 

3 3 

D3 E E 

Out EEDDDDDEDDDDD . DDDEDDDDDEDDDDDEDDDDDEE 

D3 D2 Dl 0 1 2 3 Out 
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***************** 



Histogram r Standardized Residual 

NExp N (* = l Cases, . : « Normal Curve) 

1 .09 Out * 

0 .17 3.00 

0 .43 2.67 

0 .99 2.33 . 

4 2.03 2.00 *:** 

1 3.71 1.67 * . 

8 6.09 1.33 *****:** 

9 8.95 1.00 ********: 
6 11.8 .67 ****** 

* 13.9 .33 ************ . 

* 14.7. .00 **************:**************** 

* 13.9 -.33 ********** 

*• 11.8 ^.67 ********** . 

* 8.95 -1.00 ********:* 

2 6.09 -1.33 ** 
0 3.71 -1.67 

5 2.03 -2.00 *:*** 
2 .99 -2.33 :* 

0 .43-2.67 
0 .17-3.00 
0 .09 Out 
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Standardized Scatterplot 

Across - *pred Down - *RESID 

Out EEDDDDDEDDDDDEDDDDDEDD . EDDDDDEDDDDDEE 



3 E E symbols: 

3 3 

3 3 Max N 

2 E ... E 

3 . 3 4.0 

3 3 : 8.0 

IE E * 17.0 

3 ... 3 

3 : .... 3 

0 E : * ... e 

3 ... 3 

3 .... 3 

Dl E . : . . e' 

3 . 3 

3 3 

D2 E . . E 

3 . 3 

3 3 

D3 E E 
Out EEDDDDDEDDDDD. ,DDEDDDDDEDDDDDEDDDDDEE 

D3 D2 Dl 0 1 2 3 Out 



Standardized Scatterplot 

Across - TIMELD Down - *RESID 

Out EEDDDDDEDDDDDE . DDEDDDDDEDDDDDEDDDDDEE 



3 E E symbols: 

3 3 

3 3 Max N 

2 E .... E 

3 . 3 1.0 

3 .:. .:. 3: 2.0 

IE : : E* 5.0 

3 •» • 

3 . . : . . : . . . . v— 3— 

0 E .... : : .***** e 

3 *....:. 3 

3 ..::..: 3 

Dl E ....*:. e 

3 : 3 

3 3 

D2 E . .. : E 

3 : 3 

3 3 

D3 E E 



Out EEDDDDDEDDDDDEDDDDDEDDDDDEDDDDDEDDDDDEE 

D3 D2 Dl 0 1 2 3 Out 
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Appendix F 
Raw Data 
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Pre- and Post- Intervention Data for Experimental Group 
with Learning Disabled 



Pre school errnum ldlerr ld2err ld3err 
Post 



0 


4 


8 


1 


4 


3 


0 


4 


3 


0 


2 


1 


0 


2 


5 


0 


3 


2 


0 


4 


6 


1 


3 


2 


0 


3 


4 


. 1 


1 


2 


0 


4 


4 


0 


3 


1 


0 


3 


7 


1 


4 


2 


0 


3 


4 


0 


3 


1 


0 


3 


4 


0 


2 


2 


0 


3 


7 


1 


3 


3 


0 


2 


0 


0 


0 


0 


0 


3 


4 


0 


2 


2 


0 


3 


7 


1 


4 


2 


0 


3 


6 


0 


4 


2 


0 


2 


0 


0 


0 


0 


0 


4 


7 


1 


4 


2 


0 


3 


8 


2 


4 


2 


0 


4 


8 


1 


3 


4 


0 


2 


4 


1 


2 


1 


0 


1 


3 


1 


2 


0 


0 


3 


5 


1 


3 


1 


0 


3 


6 


1 . 


3 


2 


0 


3 


6 


1 


3 


2 


0 


1 


0 


0 


0 


0 


0 


4 


7 


2 


3 


2 


0 


1 


3 


0 


2 


1 


0 


4 


7 


1 


4 


2 


0 


1 


3 


0 


2 


1 


0 


4 


3 


0 


2 


1 


0 


2 


5 


1 


3 


1 


0 


3 


8 


2 


4 


2 


0 


3 


0 


0 


0 


0 


0 


1 


.6 


2 


3 


1 


0 


1 


4 


1 


2 


1 


0 


1 


6 


2 


2 


2 


0 


1 


13 


3 


. 7 


3 


0 


2 


9 


1 


5 


3 


0 


1 


6 


1 


3 


2 


0 


4 


5 


1 


3 


1 


0 


1 


0 


0 


0 


0 


0 


2 


3 


0 


2 


1 


0 


3 


3 


0 


1 


2 
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Pre- and Post- Intervention Data for Experimental Group 
with IEP Development 



Pre school errnum mclerr mc2err mc3err 
Post 



0 


1 


6 


3 


2 


1 


0 


4 


5 


3 


1 


1 


0 


4 


4 


2 


1 


1 


0 


4 


5 


2 


3 


0 


0 


1 


4 


2 


1 


1 


0 


4 


5 


2 


1 


2 


0 


4 


7 


3 


2 


2 


0 


4 


6 


3 


1 


2 


0 


4 


4 


1 


2 


1 


0 


1 


5 


3 


1 


1 


0 


1 


6 


3 


2 


1 


0 


3 


5 


3 


1 


1 


0 


3 


4 


2 


1 


1 


0 


3 


5 


3 


1 


1 


0 


3 


5 


3 


1 


1 


0 


4 


4 


2 


1 


1 


0 


1 


4 


2 


2 


0 


0 


1 


3 


2 


1 


0 


0 


1 


4 


2 


2 


0 


0 


1 


4 


2 


1 


1 


0 


1 


0 


4 


2 


1 


0 


1 


2 


1 


1 


0 


0 


1 


3 


1 


1 


1 


0 


3 


5 


2 


2 


1 


0 


2 


3 


1 


1 


1 


0 ' 


4 


6 


4 


.1 


1 


0 


2 


3 


1 


1 


1 


0 


3 


6 


2 


3 


1 


0 


3 


3 


1 


1 


1 


0 


2 


5 


2 


2 


1 


0 


3 


3 


2 


1 


0 


0 


2 


4 


2 


1 


1 


0 


2 


3 


1 


1 


1 


0 


3 


4 


2 


1 


1 


0 


2 


3 


1 


1 


1 


0 


2 


6 


3 


1 


2 


0 


2 


3 


1 


1 


1 


0 


2 


5 


2 


2 


1 


0 


2 


3 


1 


1 


1 


0 


. 2 


6 


3 


I 


2 


0 


2 


3 


2 


1 


0 


0 


1 


5 


2 


2 


1 



99 



1 


3 


2 


1 


1 


0 


1 


3 . 


2 


1 


1 


0 


1 


4 


1 


0 


0 


1 


1 


1 


2 


0 


1 


1 


1 


3 


1 


0 


1 


0 


1 


4 


0 


0 


0 


0 


1 


4 


2 


1 


1 


0 


1 


3 


1 


0 


1 


0 


1 


1 


2 


1 


1 


0 


1 


1 


2 


1 


0 


1 


1 


2 


2 


1 


1 


0 


1 


2 


2 


1 


1 


0 


1 


2 


2 


1 


1 


0 


1 


1 


2 


1 


0 


1 


1 


2 


2 


. 1 


1 


0 


1 


1 


2 


1 


1 


0 


1 


4 


0 


0 


0 


0 


1 


3 


1 


0 


1 


0 


1 


1 


2 


1 


1 


0 


1 


4 


1 


1 


0 


0 


1 


4 


0 


0 


0 


0 


1 


4 


0 


0 


0 


0 


1 


1 


2 


1 


1 


0 


1 


4 


0 


0 


0 


0 


1 


1 


1 


0 


0 


0 


"1 


4 


0 


0 


0 


0 


1 


4 


1 


0 


0 


0 


1 


2 


2 


1 


1 


0 


1 


3 


1 


0 


1 


0 


1 


1 


2 


1 


1 


0 


1 


1 


1 


0 


1 


0 


1 


3 


1 


0 


1 


0 
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Pre- and Post- Intervention Data for Experimental Group 
with Intellectually Disabled 



Pre school errnum ihlerr ih2err 
Post 



0 


2 


6 


2 


4 


n 


2 


6 


1 


5 


0 


2 


3 


0 


3 


0 


2 


7 


3 


4 


0 


2 


6 


3 


3 


0 


2 


4 


0 


4 


0 


2 


4 


0 


4 


0 


2 


6 


2 


4 


0 


2 


7 


2 


5 


0 


2 


4 


2 


2 


0 


1 


5 


1 


4 


0 


2 


6 


2 


4 


0 


2 


5 


2 


3 


0 


2 


5 


3 


2 


0 


2 


4 


2 


2 


0 


2 


3 


0 


3 


0 


2 


2 


0 


2 


0 


1 


6 


2 


4 


0 


2 . 


2 


0 


2 


0 


2 


2 


0 


2 


0 


2 


6 


2 


4 


0 


2 


5 


1 


4 


0 


3 


4 


0 


4 


0 


3 


. 7 


2 


5 


0 


2 


5 


1 


4 


0 


2 


4 


2 


2 


0 


3 


6 


3 


3 


1 


2 


1 


0 


1 


1 


2 


3 


1 


2 


1 


3 


2 


1 


1 


1 


3 


2 


1 


1 


1 


2 


2 


1 


1 


1 


1 


2 


1 


1 


1 


3 


1 


0 


1 


1 


2 


2 


0 


2 


1 


3 


3 • 


0 


2 


1 


2 


2 


1 


1 


1 


3 


2 


0 


2 


1 


1 


2 


1 


1 


1 


2 


1 


0 


1 
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